Course Notes Advanced DSP

1
Advanced Digital Signal Processing

Abdellatif Zaidi
Department of Electrical Engineering University of Notre Dame azaidi@nd.edu
6. Spectral estimation Periodogram, Bartletts method, Welchs method, Blackman-Tukey method ARMA modeling, Yule-Walker equation and solution
Outline: 1. Introduction 2. Digital processing of continuous-time signals Retition: Sampling and sampling theorem Quantization AD- and DA-conversion 3. DFT and FFT Leakage effect Windowing FFT structure 4. Digital lters FIR-lters: Structures, linear phase lters, least-squares frequency domain design, Chebyshev approximation IIR-lters: Structures, classical analog lowpass lter approximations, conversion to digital transfer functions Finite word-length effects 5. Multirate digital signal processing Decimation and interpolation Filters in sampling rate alteration systems Polyphase decomposition and efcient structures Digital lter banks
Parts of this textbook have been realized in close collaboration with Dr. Joerg Kliewer whom I warmly thank.
Literature J. G. Proakis, D. G. Manolakis: Digital Signal Processing: Principles, Algorithms, and Applications, Prentice Hall, 2007, 4th edition S. K. Mitra: Digital Signal Processing: A ComputerBased Approach, McGraw Hill Higher Education, 2006, 3rd edition A. V. Oppenheim, R. W. Schafer: Discrete-time signal processing, Prentice Hall, 1999, 2nd edition M. H. Hayes: Statistical Signal Processing and Modeling, John Wiley and Sons, 1996 (chapter 6).
1. Introduction
1.1 Signals, systems and signal processing What does Digital Signal Processing mean? Signal:
Numerical integration and differentiation Determination of mean value, correlation, p.d.f., . . .
Physical quantity that varies with time, space or any other independent variable Mathematically: Function of one or more independent variables, s1(t) = 5 t, s2(t) = 20 t2 Examples: Temperature over time t, brightness (luminance) of an image over (x, y ), pressure of a sound wave over (x, y, z ) or (x, y, z, t) Speech signal:
1 Amplitude 0.5 0 x 10
4
Properties of the system (e.g. linear/nonlinear) determine the properties of the whole processing operation System: Denition also includes: software realizations of operations on a signal, which are carried out on a digital computer ( software implementation of the system) digital hardware realizations (logic circuits) congured such that they are able to perform the processing operation, or most general denition: a combination of both
Digital signal processing: Processing of signals by digital means (software and/or hardware) Includes: Conversion from the analog to the digital domain and back (physical signals are analog)
0.5 1 0.1 0.2 0.3 0.4 t [s] 0.5 0.6 0.7
Signal Processing:
Mathematical specication of the processing operations Algorithm: method or set of rules for implementing the system by a program that performs the corresponding mathematical operations Emphasis on computationally efcient algorithms, which are fast and easily implemented.
Passing the signal through a system
Examples: Modication of the signal (ltering, interpolation, noise reduction, equalization, . . .) Prediction, transformation to another domain (e.g. Fourier transform)
3 4
Basic elements of a digital signal processing system Analog signal processing:

Analog input signal Analog signal processor Analog output signal
1.2 Digital signal processors (DSPs)
Programmable microprocessor (more exibility), or hardwired digital processor (ASIC, application specic integrated circuit) (faster, cheaper)
Often programmable DSPs (simply called DSPs) are used for evaluation purposes, for prototypes and for complex algorithms:
Digital signal processing:

(A/D: Analog-to-digital, D/A: Digital-to-analog)
Analog input signal Digital input signal Digital signal processor Digital output signal Analog output signal
A/D converter
D/A converter
Fixed-point processors: Twos-complement number representation. Floating-point processors: Floating point number representation (as for example used in PC processors)
Overview over some available DSP processors see next page. Performance example: 256-point complex FFT
Why has digital signal processing become so popular? Digital signal processing has many advantages compared to analog processing:
Property Dynamics Precision Digital only limited by complexity generally unlimited (costs, complexity precision) without problems low Analog generally limited generally limited (costs increase drastically with required precision) problematic higher
Aging Production costs Frequency range Linear-phase frequency responses Complex algorithms
dmin amin , dmax amax

exactly realizable approximately realizable
realizable
strong limitations
(from [Evaluating DSP Processor Performance, Berkeley Design Technology, Inc., 2000])
Some currently available DSP processors and their properties (2006):

Manufacturer Analog Devices Family ADSP-219x ADSP-2126x ADSP-BF5xx ADSP-TS20x DSP563xx DSP568xx MSC71xx MSC81xx TMS320C24x TMS320C54x TMS320C55x TMS320C64x TMS320C67x Arithmetic xed-point oating-point xed-point oating/xed-point xed-point xed-point xed-point xed-point xed-point xed-point xed-point xed-point oating-point Data width (bits) 16 32/40 16 8/16/32/40 24 16 16 16 16 16 16 8/16 32 BDTImark 2000(TM) 410 1090 4190 6400 820 110 3370 5610 n/a 500 1460 9130 1470 Core clock speed 160 MHz 200 MHz 750 MHz 600 MHz 275 MHz 80 MHz 300 MHz 500 MHz 40 MHz 160 MHz 300 MHz 1 GHz 300 MHz Unit price qty. 10000 $11-26 $5-15 $5-32 $131-205 $4-47 $3-12 $13-35 $77-184 $2-8 $3-54 $4-17 $15-208 $12-31
Freescale
Texas-Instuments
2. Digital Processing of Continuous-Time Signals

Digital signal processing system from above is rened:
Anti-aliasing lowpass lter Sample-andhold circuit A/D Digital signal processor
Lowpass reconstruction lter
Sample-andhold circuit
D/A
2.1 Sampling
Generation of discrete-time signals from continuous-time signals

Ideal sampling Ideally sampled signal xs(t) obtained by multiplication of the continuous-time signal xc(t) with the periodic impulse train
n=
s(t) =
(t nT ),
where (t) is the unit impulse function and T the sampling period:
n=
xs(t) = xc(t) =
n=
(t nT )
(2.1)
xc(nT ) (t nT )
(2.2)
10
(sifting property of the impulse function)

xc (t) (a) t
we nally have for the Fourier transform of xs(t)

k=
Xs(j ) =
1 T
Xc(j ( ks)).
(2.4)
s(t) (b)
1
(lengths of Dirac deltas correspond to their weighting)
Periodically repeated copies of Xs(j )/T , shifted by integer multiples of the sampling frequency
Xc (j )
t
0 T 2T 3T 4T 5T
.
(a)
xs (t) (c)
(lengths of Dirac deltas correspond to their weighting)

N N
S (j )
t
.
(b)
2 T
2T
3T
4T
5T
3s 2s s 0 s 2s 3s
How does the Fourier transform F {xs(t)} = Xs(j ) look like? Fourier transform of an impulse train:
Xs (j ) (c)
1 T
s(t) S (j ) =
2 ( ks) T k=
3s
2s
s N
(2.3)
6 4 2 0
s 2s (s N ) 2 4
3s
= T Xs (j ) (d)
1 T
s = 2/T : sampling frequency in radians/s. Writing (2.1) in the Fourier domain, Xs(j ) = 1 Xc(j ) S (j ), 2
11
= Aliasing
s 2s (s N )
12
(a) Fourier transform of a bandlimited continuous-time input signal Xc(j ), highest frequency is N (b) Fourier transform of the Dirac impulse train (c) Result of the convolution S (j ) Xc(j ). It is evident that when s N > N or s > 2N (2.5) the replicas of Xc(j ) do not overlap. xc(t) can be recovered with an ideal lowpass lter!
a0(t) denotes the rectangular prototype window: a0(t) = rect t T /2 T 0 for |t| > 1/2
(2.7)
with rect(t) :=

1 for |t| < 1/2

rect(t) sinc(/2), sinc(x) := sin(x)/x
(2.8)
(d) If (2.5) does not hold, i.e. if s 2N , the copies of Xc(j ) overlap and the signal xc(t) cannot be recovered by lowpass ltering. The distortions in the gray shaded areas are called aliasing distortions or simply aliasing.
Also in (c): Representation with the discrete (normalized) frequency
Fourier transform of an(t): Fourier transform of the rectangular time window in (2.7) (see properties of the Fourier transform)
= T = 2f T (f frequency in Hz) for the discrete signal xc (nT ) = x(n)

with F {x(n)} = X (ej ), F {} denoting the Fourier transform for discrete-time aperiodic signals (DTFT)
Nonideal sampling
A0(j ) = F {a0(t)} = T sinc(T /2) e

Fourier transform of an(t) in (2.6) (analog to (2.3)):
j T /2
(2.9)
Modeling the sampling operation with the Dirac impulse train is not a feasible model in real life, since we always need a nite amount of time for acquiring a signal sample.
Nonideally sampled signal xn(t) obtained by multiplication of a continuous-time signal xc(t) with a periodic rectangular window function an(t): xn(t) = xc(t) an(t) where
An(j ) = A0(j ) = 2
2 ( ks) T k=
sinc(ks T /2) e
jk jks T /2
k=
( ks)
an(t) = a0(t)
n=
(t n T ) =
n=
a0(t nT )
(2.6)
13
= 2
k=
sinc(k) e
( ks)
(2.10)
14
Since
xn(t) = xc(t) an(t) Xn(j ) =

we nally have by inserting (2.10)
1 (Xc(j )An(j )) 2
Solution: Sampling is performed by a sample-and-hold (S/H) circuit
Xn(j ) =
k=
sinc(k) e
jk
Xc(j ( ks)).
(2.11)
From (2.11) we can deduce the following:
Compared to the result in the ideal sampling case (cp. (2.4)) here each repeated spectrum at the center frequency ks is weighted with the term sinc(k) ejk. The energy |Xn(j )|2 is proportional 2: This is problematic since in order to approximate the ideal case we would like to choose the parameter as small as possible.
(from [Proakis, Manolakis, 1996])
(a) Basic elements of an A/D converter, (b) time-domain response of an ideal S/H circuit
The goal is to continously sample the input signal and to hold that value constant as long as it takes for the A/D converter to obtain its digital representation. Ideal S/H circuit introduces no distortion and can be modeled as an ideal sampler. Drawbacks for the nonideal sampling case can be avoided (all results for the ideal case hold here as well).
15
16
2.2 Sampling Theorem Reconstruction of an ideally sampled signal by ideal lowpass ltering:
s(t) =
n=
Then, with the denition of the rect() function in (2.8) we have
(t nT )
Hr (j ) = T rect(/s) hr (t) = sinc(st/2). (2.14) Combining (2.13), (2.14), and (2.2) yields
(a)
xc (t) xs (t)
Hr (j )
xr (t)
Hr (j ) (d)
T N < c < (s N )
xc(t) =
Xc (j ) (b)
1
n=
xc(nT ) ( nT ) sinc
1 s (t ) 2 1 s (t ) 2 .
c c
xc(nT )
N N
Xr (j ) (e)
1
n=
( nT ) sinc 1 s(t nT ) 2
Xs (j ) (c)
1 T
=

N N
n=
xc(nT ) sinc
2s
s N
s 2s (s N )
In order to get the input signal xc(t) back after reconstruction, i.e. Xr (j ) = Xc(j ), the conditions
Sampling theorem: Every bandlimited continuous-time signal xc(t) with N < s/2 can be uniquely recovered from its samples xc(nT ) according to
n=
N <
s 2
and
N < c < (s N )
(2.12)
xc(t) =
xc(nT ) sinc
1 s(t nT ) 2
(2.15)
have both to be satised. Then,
Xc(j ) = Xr (j ) = Xs(j ) Hr (j )
Remarks:
xc(t) = xr (t) = xs(t) hr (t). (2.13)

We now choose the cutoff frequency c of the lowpass lter as c = s/2 (satises both inequalities in (2.12)).
17
Eq. (2.15) is called the ideal interpolation formula, and the sinc-function is named ideal interpolation function
18
Reconstruction of a continuous-time signal using ideal interpolation:
S/H circuit serves as a deglitcher: Output of the D/A converter is held constant at the previous output value until the new sample at the D/A output reaches steady state
Anti-aliasing lowpass ltering: In order to avoid aliasing, the continuous-time input signal has to be bandlimited by means of an anti-aliasing lowpass-lter with cut-off frequency c s/2 prior to sampling, such that the sampling theorem is satised. 2.3 Reconstruction with sample-and-hold circuit In practice, a reconstruction is carried out by combining a D/A converter with a S/H circuit, followed by a lowpass reconstruction (smoothing) lter
digital input signal
D/A
S/H h0 (t)
x0 (t)
Lowpass hr (t)
xDA (t)
D/A converter accepts electrical signals that correspond to binary words as input, and delivers an output voltage or current being proportional to the value of the binary word for every clock interval nT Often, the application on an input code word yields a high-amplitude transient at the output of the D/A converter (glitch)
19
Analysis: The S/H circuit has the impulse response
h0(t) = rect
t T /2 T
(2.16)
which leads to a frequency response
H0(j ) = T sinc(T /2) e
j T /2
(2.17)
20
No sharp cutoff frequency response characteristics we have undesirable frequency components (above s/2), which can be removed by passing x0(t) through a lowpass reconstruction lter hr (t). This operation is equivalent to smoothing the staircase-like signal x0(t) after the S/H operation. When we now suppose that the reconstruction lter hr (t) is an ideal lowpass with cutoff frequency c = s/2 and an amplication of one, the only distortion in the reconstructed signal xDA(t) is due to the S/H operation:
(a)
|Xs (j )|
1 T
|Xc (j )|
1 T
s 0 s 2s
(b)
T sinc(T /2)
T
s 0 s 2s
(c)
|X0 (j )|
1
|XDA(j ) = |Xc(j )| |sinc(T /2)|
(2.18)
s
0 s 2s
|Xc(j )| denotes the magnitude spectrum for the ideal reconstruction case.
(d)
|Hr (j )|
1 c = s /2
Additional distortion when the reconstruction lter is not ideal (as in real life!) Distortion due to the sinc-function may be corrected by prebiasing the frequency response of the reconstruction lter
(e)
s c 0 c s
|XDA (j )|
1
Spectral interpretation of the reconstruction process (see next page): (a) Magnitude frequency response of the ideally sampled continuous-time signal (b) Frequency response of the S/H circuit (phase factor ej T /2 omitted) (c) Magnitude frequency response after the S/H circuit (d) Magnitude frequency response: lowpass reconstruction lter (e) Magnitude frequency response of the reconstructed continuous-time signal
21
s
0 s
22
2.4 Quantization Conversion carried out by an A/D converter involves quantization of the sampled input signal xs(nT ) and the encoding of the resulting binary representation
Example: midtreat quantizer with L = 8 levels and range R = 8

4 d1 = 3 2 0 2 3 d9 = x1 d2 x2 d3 x3 d4 x4 d5 x5 d6 x6 d7 x7 d8 x8 Amplitude
Quantization is a nonlinear and noninvertible process which realizes the mapping xs(nT ) = x(n) xk I ,
(2.19)
Range R of quantizer
Signal amplitude range is divided into L intervals In using the L +1 decision levels d1, d2, . . . , dL+1: In = {dk < x(n) dk+1},
Quantization Decision levels levels
where the amplitude xk is taken from a nite alphabet I .
Quantization error e(n) with respect to the unquantized signal (2.21) < e(n) 2 2 If the dynamic range of the input signal (xmax xmin) is larger than the range of the quantizer, the samples exceeding the quantizer range are clipped, which leads to e(n) > /2. Quantization characteristic function for a midtreat quantizer with L = 8:
k = 1, 2, . . . , L
Ik dk xk dk+1
d3 x3 d4 x4 d5 Amplitude
Mapping in (2.19) is denoted as x (n) = Q[x(n)]
Uniform or linear quantizers with constant quantization step size are very often used in signal processing applications: = xk+1 xk = const.,
for all k = 1, 2, . . . , L 1 (2.20)
Midtreat quantizer: Zero is assigned a quantization level Midrise quantizer: Zero is assigned a decision level
23
24
Coding The coding process in an A/D converter assigns a binary number to each quantization level.
signicant bit (LSB), has the value
With a wordlength of b bits we can represent 2 > L binary numbers, which yields b log2(L).
(2.22)
x = 0 +
b1 =1
(2.24)
Number representation has no inuence on the performance of the quantization process!

Performance degradations in practical A/D converters:
The step size or the resolution of the A/D converter is given as = R 2b

with the range R of the quantizer. (2.23)
Commonly used bipolar codes:
Twos complement representation is used in most xedpoint DSPs: A b-bit binary fraction [012 . . . b1], 0 denoting the most signicant bit (MSB) and b1 the least
25
26
Quantization errors Quantization error is modeled as noise, which is added to the unquantized signal:
Actual system
x(n)
Quantizer Q(x(n))
e(n)
2 2 where x denotes the signal power and e the power of the quantization noise. Quantization noise is assumed to be uniformly distributed in the range (/2, /2):
x (n)
Mathematical model
x(n)
x (n) = x(n) + e(n)

Zero mean, and a variance of
2 e
Assumptions:
The quantization error e(n) is uniformly distributed over the range 2 < e(n) < 2 .
The error sequence e(n) is modeled as a stationary white noise sequence.
1 = e p(e) de = /2
2
/2
2 e de = 12 /2
/2 2
(2.26) Inserting (2.23) into (2.26) yields
The error sequence e(n) is uncorrelated with the signal sequence x(n). The signal sequence is assumed to have zero mean.
Assumptions do not hold in general, but they are fairly well satised for large quantizer wordlengths b. Effect of the quantization error or quantization noise on the resulting signal x (n) can be evaluated in terms of the signal-tonoise ratio (SNR) in Decibels (dB)
2 x 2 e
e =
22b R2 , 12
(2.27)
and by using this in (2.25) we obtain SNR = 10 log10

2 x 2 e
= 10 log10 R x
2 12 22b x R2
= 6.02 b + 10.8 20 log10

SNR := 10 log10
dB.
(2.28)
(2.25) Term () in (2.28):

27
( )
28
x root-mean-square (RMS) amplitude of the signal v (t) x to small decreasing SNR Furthermore (not directly from ()): x to large range R is exceeded Signal amplitude has to be carefully matched to the range of the A/D converter For music and speech a good choice is x = R/4. Then the SNR of a b-bit quantizer can be approximately determined as
SNR = 6.02 b 1.25 dB. (2.29)
2.5 Analog-to-digital converter realizations Flash A/D converter
Analog comparator:
Each additional bit in the quantizer increases the signal-to-noise ratio by 6 dB! Examples: Narrowband speech: b = 8 Bit SNR = 46.9 dB Music (CD): b = 16 Bit SNR = 95.1 dB Music (Studio): b = 24 Bit SNR = 143.2 dB
(from [Mitra, 2000], N = b: resolution in bits)
Analog input voltage VA is simultaneously compared with a set of 2b 1 separated reference voltage levels by means of a set of 2b1 analog comparators locations of the comparator circuits indicate range of the input voltage. All output bits are developed simultaneously very fast conversion Hardware requirements increase exponentially with an increase in resolution Flash converters are used for low-resultion (b < 8 bit) and high-speed conversion applications.
Serial-to-parallel A/D converters Here, two b/2-bit ash converters in a serial-parallel conguration are used to reduce the hardware complextity at the expense of a slightly higher conversion time
29 30
Subranging A/D converter:
approximation in the shift register is converted into an (analog) reference voltage VD by D/A conversion (binary representation [01 . . . k k+1 . . . b1], k {0, 1}k):
Case 1: Reference voltage VD < VA increase the binary number by setting both the k-th bit and the (k +1)-th bit to 1 Case 2: Reference voltage VD VA decrease the binary number by setting the k-th bit to 0 and the (k +1)-th bit to 1 High resolution and fairly high speed at moderate costs, widely used in digital signal processing applications
Oversampling sigma-delta A/D converter, to be discussed in Section 5. . .
Ripple A/D converter:
2.6 Digital-to-analog converter realizations Weighted-resistor D/A converter

Advantage of both structures: Always one converter is idle while the other one is operating Only one b/2-bit converter is necessary Sucessive approximation A/D converter
Output Vo of the D/A converter is given by
Vo =
N 1 =0
(2N
RL VR 1)RL + 1
Iterative approach: At the k-th step of the iteration the binary

31
VR : reference voltage Full-scale output voltage Vo,F S is obtained when = 1 for all = 0, . . . , N 1:
32
Vo,F S
(2N 1)RL N = N VR VR , since (2 1)RL 1 (2 1)RL + 1
3. DFT and FFT

3.1 DFT and signal processing Denition DFT from Signals and Systems:
N 1 n=0 N 1 k=0
Disadvantage: For high resolutions the spread of the resistor values becomes very large Resistor-ladder D/A converter
DFT: v (n) VN (k) =
v ( n) WN
kn
(3.1)
1 IDFT: VN (k) v (n) = N
VN (k) WN
kn
(3.2)
with WN := ej 2/N , N : number of DFT points 3.1.1 Linear and circular convolution Linear convolution of two sequences v1(n) and v2(n), n Z Z:
R2R ladder D/A converter, most widely used in practice. Output Vo of the D/A converter: Vo =
N 1 =0
VR RL N 1 2(RL + R) 2
In practice, often 2RL R, and thus, the full-scale output voltage Vo,F S is given as
y l ( n ) = v1 ( n ) v2 ( n ) = v2 ( n ) v1 ( n ) =
k=
Vo,F S
(2N 1) VR 2N
v1 ( k ) v2 ( n k ) =
k=
v2(k) v1(n k) (3.3)
Oversampling sigma-delta D/A converter, to be discussed in Section 5. . .
Circular convolution of two periodic sequences v1(n) and v2(n), n = {0, . . . , N1,2 1} with the same period N1 = N2 = N and n0 Z Z:
33 34
Substitution of the DFT denition in (3.1) for v1(n) and v2(n):
y c ( n ) = v1 ( n ) v2 ( n ) = v2 ( n ) v1 ( n ) =
n0 +N 1 k =n0
y ( n) =
v1 ( k ) v2 ( n k ) =
n0 +N 1 k =n0
1 N 1 N
N 1 k=0 N 1 m=0
N 1 m=0
v1(m)WN
N 1 l=0
km
N 1 l=0
v2 ( l ) W N
kl
WN
kn
v2 ( k ) v1 ( n k )
(3.4)
v1(m)
v2 ( l )
N 1 k=0
WN
k(nml)
(3.7) We also use the symbol

N
instead of .
DFT and circular convolution Inverse transform of a nite-length sequence v (n), n, k = 0, . . . , N 1:
Term in brackets: Summation over the unit circle

N 1 k=0
j 2k(nml)/N
N 0
for
otherwise
l = n m + N, Z Z
(3.8)
v (n) VN (k) v (n) = v (n + N )
(3.5)
Substituting (3.8) into (3.7) yields the desired relation
DFT of a nite-length sequence and its periodic extension is identical! Circular convolution property (n, k = 0, . . . , N 1) (v1(n) and v2(n) denote nite-length sequences): y ( n ) = v1 ( n )
Proof:
N
y ( n) =
N 1 k=0
v1 ( k )
v2(n k + N )
=: v2 ((nk))N (periodic extension)
v2(n) Y (k) = V1N (k) V2N (k)

(3.6)
N 1 k=0 N 1 k=0
N 1 k=0
v1(k)v2((n k))N
N
(3.9)
= v1 ( n ) Y (k )WN
kn
v2 ( n )
1 IDFT of Y (k): y (n) = N 1 = N
V1N (k)V2N (k)WN
kn
35
36
Example: Circular convolution y (n) = v1(n)

v2 (n)
v2 ( n ) :
operation for linear ltering is linear convolution. How can this be achieved by means of the DFT? Given: Finite-length sequences v1(n) with length N1 and v2(n) with length N2
n
0 N
Linear convolution: y ( n) =
n
v1 (n) = (n 1)
N1 1 k=0
v1 ( k ) v2 ( n k )
v2 ((0 k ))N , k = 0, . . . , N 1, n = 0
k
0 N
v2 ((1 k ))N , k = 0, . . . , N 1, n = 1
k
0 N
Length of the convolution result y (n): N1 + N2 1 Frequency domain equivalent: Y (ej ) = V1(ej ) V2(ej ) In order to represent the sequence y (n) uniquely in the frequency domain by samples of its spectrum Y (ej ), the number of samples must be equal or exceed N1 + N2 1 DFT of size N N1 + N2 1 is required. Then, the DFT of the linear convolution y (n) = v1(n) v2(n) is Y (k) = V1(k) V2(k), k = 0, . . . , N 1. This result can be summarized as follows: The circular convolution of two sequences v1(n) with length N1 and v2(n) with length N2 leads to the same result as the linear convolution v1(n) v2(n) when the lengths of v1(n) and v2(n) are increased to N = N1 + N2 1 points by zero padding. Other interpretation: Circular convolution as linear convolution with aliasing IDFT leads to periodic sequence in the time-domain: y (n N ) n = 0, . . . , N 1, yp(n) = = 0 otherwise
38
y (n) = v1 (n) N v2 (n)
n
0 N
3.1.2 Use of the DFT in linear ltering
Filtering operation can also be carried out in the frequency domain using the DFT attractive since fast algorithms (fast Fourier transforms) exist DFT only realizes circular convolution, however, the desired
37
with Y (k) = DFTN {y (n)} = DFTN {yp(n)}
3.1.3 Filtering of long data sequences Filtering of a long input signal v (n) with the nite impulse response h(n) of length N2 Overlap-add method 1. Input signal is segmented into separate blocks: v (n) = v (n + N1), n {0, 1, . . . , N11}, Z Z. 2. Zero-padding for the signal blocks v (n) v (n) and the impulse response h(n) h(n) to the length N = N1 + N2 1. Input signal can be reconstructed according to
For N < N1 + N2 1: Circular convolution equivalent to linear convolution followed by time domain aliasing Example:
1
0 N1 = N2
x1 (n) = x2 (n) N1 = N2 = 6 n
N1
y (n) = x1 (n) x2 (n)
n
0 2N1 1
v ( n) =
y (n N1 ), N1 = 6
N1
v (n N1)
n
0 N1
since v (n) = 0 for n = N1 +1, . . . , N . 3. The two N -point DFTs are multiplied together to form
N1
y (n + N1 ), N1 = 6
(k) H (k ), Y (k) = V
k = 0, . . . , N 1.
n
N1 0
N1
x1 (n) N x2 (n) N1 = N2 = 6 n
0 N1 = N2
4. The N -point IDFT yields data blocks that are free from aliasing due to the zero-padding in step 2. 5. Since each input data block v (n) is terminated with N N1 zeros the last N N1 points from each output block y (n) must be overlapped and added to the rst N N1 points of the succeeding block (linearity property of convolution):
N1
x1 (n) N x2 (n) N = 12 n
y ( n) = Overlap-add method
39
y (n N1)
2N1 1
40
Linear FIR (nite impulse response) ltering by the overlap-add method:

Input signal:
L L L
4. Since the input signal block is of length N the rst N N1 points are corrupted by aliasing and must be discarded. The last N2 = N N1 1 samples in y (n) are exactly the same as the result from linear convolution. 5. In order to avoid the loss of samples due to aliasing the last N N1 samples are saved and appended at the beginning of the next block. The processing is started by setting the rst N N1 samples of the rst block to zero.
x 0 (n)
11 00 00 11
N N1 zeros x 1 (n)
11 00 00 11
Linear FIR ltering by the overlap-save method:

Input signal:
N N1 zeros x 2 (n)
11 00 00 11
N N1 zeros
Output signal:
y0 (n)
11 00 00 11 00 11 00 11
N N1 samples added together y1 (n)
N N1 zeros
NL
11 00 00 11 00 11 00 11
x0 (n)
11 00 00 11 00 11 00 11
N N1 samples added together y2 (n)
11 00 00 11
11 00 00 11 00 11 00 11
x1 (n)
Output signal:
11 00 00 11 00 11 00 11
x2 (n)
11 00 00 11
Overlap-save method 1. Input signal is segmented into by N N1 samples overlapping blocks: v (n) = v (n + N1), n {0, 1, . . . , N 1}, Z Z. (n) to 2. Zero-padding of the lter impulse response h(n) h 3. The two N -point DFTs are multiplied together to form (k ), k = 0 , . . . , N 1 . Y (k) = V (k) H
41
11 00 00 11
y0 (n)
Discard N N1 samples
11 00 00 11
y1 (n)
11 00 00 11
y2 (n)
the length N = N1 + N2 1.
Nore computationally efcient than linear convolution? Yes, in combination with very efcient algorithms for DFT computation.
42
3.1.4 Frequency analysis of stationary signals Leakage effect Spectral analysis of an analog signal v (t):
We nally have
Antialiasing lowpass ltering and sampling with s 2b, b denoting the cut-off frequency of the signal For practical purposes (delay, complexity): Limitation of the signal duration to the time interval T0 = L T (L: number of samples under consideration, T : sampling interval)
Limitation to a signal duration of T0 can be modeled as multiplication of the sampled input signal v (n) with a rectangular window w(n)
(ej ) = 1 V (ej ) W (ej ) V 2 1 j ( 0 ) j ( +0 ) W (e ) + W (e ) = 2 (ej )| for L = 25: Nagnitude frequency response |V
(3.13)
(from [Proakis, Nanolakis, 1996])
v (n) = v (n) w(n) with w(n) =
0 otherwise.
1 for 0 n L 1,
(3.10) Suppose that the input sequence just consists of a single sinusoid, that is v (n) = cos(0n). The Fourier transform is
(ej ) is not localized to one frequency, Windowed spectrum V instead it is spread out over the whole frequency range spectral leaking
V (e ) = ( ( 0) + ( + 0)).
(3.11)
First zero crossing of W (ej ) at z = 2/L: The larger the number of sampling points L (and thus also the width of the rectangular window) the smaller becomes z (and thus also the main lobe of the frequency response).
For the window w(n) the Fourier transform can be obtained as

L1 n=0
Decreasing the frequency resolution leads to an increase of the time resolution and vice versa (duality of time and frequency domain).
In practice we use the DFT in order to obtain a sampled (ejk ), representation of the spectrum according to V k = 0, . . . , N 1.
44
W (e ) =
jn
1 ejL j (L1)/2 sin(L/2) = . =e j 1e sin(/2) (3.12)

43
Special case: If
N = L and 0 =
2 , N
= 0, . . . , N 1
then the Fourier transform is exactly zero at the sampled frequencies k , except for k = . Example: (N = 64, n = 0, . . . , N 1, rectangular window w ( n) )
k = k2/N except for k = 5, since 0 is exactly an integer multiple of 2/N periodic repetition of v0(n) leads to a pure cosine sequence Right hand side: slight increase of 0 = 2/N for Z Z 1(ejk ) = 0 for k = k2/N , periodic repetition is V not a cosine sequence anymore
Windowing and different window functions Windowing not only distorts the spectral estimate due to leakage effects, it also reduces the spectral resolution. Consider a sequence of two frequency components v (n) = cos(1n)+cos(2n) with the Fourier transform
v0(n) = cos 5
2 n , v1(n) = cos N
1
2 + N N
v0(n)=cos(2/N 5n)
1
v1(n)=cos((2/N 5+/N) n)
0.5
0.5
V (e ) =
1 j ( 1 ) j ( 2 ) W (e ) + W (e )+ 2 + W (e
j ( +1 )
) + W (e
j ( +2 )
0.5
0.5
20
40
60
20
40
60
n DFT(v0(n) w(n)), rect. window

1 0.8 0.6 0.4 0.2 0 1 0.8 0.6 0.4 0.2 0
n DFT(v1(n) w(n)), rect. window
where W (ej ) is the Fourier transform of the rectangular window from (3.12).
2/L < |12|: Two maxima, main lobes for both window spectra W (ej (1)) and W (ej (2)) can be separated |1 2| = 2/L: Correct values of the spectral samples, but main lobes cannot be separated anymore |1 2| < 2/L: Nain lobes of W (ej (1)) and W (ej (2)) overlap
30
10
20
30
10
20
0(ejk ) = V0(ejk ) W (ejk ) = 0 for Left hand side: V

45
Ability to resolve spectral lines of different frequencies is limited by the main lobe width, which also depends on the length of the window impulse response L.
46
Example: Magnitude frequency response |V (ej )| for
window, specied as
v (n) = cos(0n) + cos(1n) + cos(2n)
(3.14)
with 0 = 0.2 , 1 = 0.22 , 2 = 0.6 and (a) L = 25, (b) L = 50, (c) L = 100
(3.15) (ej )| from (3.13), where Nagnitude frequency response |V W (ej ) is replaced by WHan(ej ) (L = 25):
1 1 cos wHan(n) = 2 0
2 L1 n
for 0 n L 1, otherwise.
Spectrum of the signal in (3.14) after it is windowed with wHan(n) in (3.15):
The cosines with the frequencies 0 and 1 are only resolvable for L = 100. To reduce leakage, we can choose a different window function with lower side lobes (however, this comes with an increase of the width of the main lobe). One choice could be the Hanning
47
48
The reduction of the sidelobes and the reduced resolution compared to the rectangular window can be clearly observed. Alternative: Hamming window
wHam (n) =
2 n 0.54 0.46 cos L 1 0
for 0 n L 1, otherwise. (3.16)
is increasing N ). However, the spectral resolution is not increased (separation of two closely adjacent sine spectra lines). An increase in resolution can only be obtained by increasing the length of the input signal to be analyzed (that is increasing L), which also results in a longer window (see examples above). 3.2 Fast computation of the DFT: The FFT Complexity of the DFT calculation in (3.1) for v (n) C I, VN (k) C I:
Comparison of rectangular, Hanning and Hamming window (L = 64):

Rectangular window wrect(n) 1 0 20 log
10
|Wrect(ej )|
0.5
50
VN (k) =
0 0.2 0.4 0.6 / 20 log10 |WHan(ej )| 0.8 1
N 1
v ( n) WN
dB
kn
n=0 1 complex multiplication

0 0 20 n 40 60 100
for k = 0, . . . , N 1
N results
N compl. mult., N compl. add.
Hanning window wHan(n) 1 0
Overall N 2 complex multiplications and additions.
0.5
50
20
40
60
100
0.2
0.4 0.6 / 20 log10 |WHam(e )|

j
0.8
Hamming window wHam(n) 1 0
0.5
50
20
40
60
100
0.2
0.4 0.6 /
0.8
Remark: Spectral analysis with DFT
Sampling grid can be made arbitrarily ne by increasing the length of the windowed signal with zero padding (that
49
Remarks: 1 complex multiplication 4 real-valued mult. + 2 realvalued additions 1 complex addition 2 real valued additions A closer evaluation reveals that there are slightly less than N 2 operations: N values have to be added up (N 1) additions. Factors ej 0, ej, ej 2 no real multiplications. For k = 0 no multiplication at all. Complexity of the the DFT becomes extremely large for large values of N (i.e. N = 1024) efcient algorithms advantageous.
50
dB
dB
Fast algorithms for DFT calculation (as the fast Fourier transform, kn FFT) exploit symmetry and periodicity properties of WN as
Since
k(Nn) kn kn complex conjugate symmetry: WN = WN = ( WN ) k ( n+N ) ( k +N ) n kn periodicity in k and n: WN = WN = WN .
WN = e
2j 2/N
=e
j 2/(N/2)
= WN/2
we can rewrite (3.17) as
3.2.1 Radix-2 decimation-in-time FFT algorithms Principle: Decomposing the DFT computation into DFT computations of smaller size by means of decomposing the N -point input sequence v (n) into smaller sequences decimation-in-time Prerequisite: N integer power of two, i.e. N = 2m, m = log2(N ) I N radix-2 Decomposing a N -point DFT into two N/2-point transforms DFT V (k) (we drop the subscript N for clarity) can be written as
V (k ) =
N/21 n=0
kn v (2n) WN/2
k WN
N/21 n=0
v (2n + 1) WN/2
(3.18)
kn
= G( k ) + WN H ( k ) ,
k = 0, . . . , N 1
(3.19)
Each of the sums in (3.18) represents a N/2 DFT over the even- and odd-numbered points of v (n), respectively. G(k) and H (k) need only to be computed for 0, . . . , N/2 1 since both are periodic with period N/2.
Signal ow graph for N = 8 (v x, V X ):
V (k ) =
N 1 n=0
v ( n) WN ,
kn
k = 0, . . . , N 1 +
N/21 n=0
N/21 n=0
2kn v (2n) WN
v (2n + 1) WN
k(2n+1)
(3.17)
(from [Oppenheim, Schafer, 1999])
where in the last step v (n) is separated into two N/2-point sequences consisting of the even- and odd-numbered points in v ( n) .
51
Complexity: 2 DFTs of length N/2 2 N 2/4 = N 2/2 operations +

52
N operations for the combination of both DFTs N + N 2/2 operations (less than N 2 for N > 2)
Decomposition into 4 N/4-point DFTs
The overall ow graph now looks like (v x, V X ):
G(k) and H (k) from (3.19) can also be written as G( k ) =

N/41 n=0
g (2n) WN/4 + WN/2
kn
N/41 n=0
g (2n +1) WN/4,

(3.20)
kn
H (k ) =
N/41 n=0
kn h(2n) WN/4
k WN/2
N/41 n=0
h(2n +1) WN/4

(3.21)
kn
where k = 0, . . . , N/2 1.
Complexity: 4 DFTs of length N/4 N 2/4 operations + 2 N/2 + N operations for the reconstruction Final step: Decomposition into 2-point DFTs DFT of length 2:
N 2/4 + 2N complex multiplications and additions
V2 (0) = v (0) + W2 v (1) = v (0) + v (1)

(3.22) (3.23)
V2 (1) = v (0) + W2 v (1) = v (0) v (1)

Flow graph:

Inserting this in the resulting structure from the last step yields the overall ow graph for (N = 8)-point FFT: (v x, V X ):
53 54
WN
N/2
= ej (2/N )N/2 = ej = 1. Therefore we have WN

+N/2
= WN WN
N/2
= WN .
Using this modication, the resulting ow graph for N = 8 is given as (v x, V X ):
In general, our decomposition requires m = log2(N ) = ld N stages and for N 1 we have
N m = N ld N
(instead of N 2)
complex multiplications and additions.
Examples: N = 32 N 2 1000, N ld N 160 factor 6 N = 1024 N 2 106, N ld N 104 factor 100 Buttery computations Basic building block of the above ow graph is called buttery ( {0, . . . , N/2 1}):
1 1
WN
In-place computations
The intermediate results VN (k1,2) in the -th stage, = 0, . . . , m 1 are obtained as VN (k1) = VN VN (k2) = VN
() () (1)
()
(k1) + WN VN
(1) (1)
( k2 ) , ( k2 )
Simplication
1
WN
(1)
1 WN
(+N/2)
(k1) WN VN
The simplication is due to the fact that

55
(buttery computations) where k1, k2, {0, . . . , N 1} vary from stage to stage.
56
Only N storage cells are needed, which rst contain the values v (n), then the results from the individual stages and nally the values VN (k) In-place algorithm
Bit-reversal
Inverse FFT According to (3.2) we have for the inverse DFT
v (n)-values at the input of the decimation-in-time ow graph in permuted order Example for N = 8, where the indices are written in binary notation:
# ow graph input v (n) # ow graph input v (n) 000 v (000) 100 v (001) 001 v (100) 101 v (101) 010 v (010) 110 v (011) 011 v (110) 111 v (111)
1 v ( n) = N
that is
N 1 k=0
VN (k) WN
kn
1 v (n) = N
N 1 k=0
VN (k) WN ,
kn
(3.24)
v ( N n) =
1 DFT{VN (k)} N
Input data is stored in bit-reversed order

Bit-reversed order is due to the sorting in even and odd indices in every stage, and thus is also necessary for in-place computation: (v x):
With additional scaling and index permutations the IDFT can be calculated with the same FFT algorithms as the DFT!
3.2.2 FFT alternatives Alternative DIT structures Rearranging of the nodes in the signal ow graphs lead to FFTs with almost arbitrary permutations of the input and output sequence. Reasonable approaches are structures (a) without bitreversal, or (b) bit-reversal in the frequency domain
57
58
(a)
(b)
Radix r and mixed-radix FFTs When we gerally use
N =r
for
r 2,
r, m I N
(3.25)
we obtain DIF or DIT decompositions with a radix r . Besides r = 2, r = 3 and r = 4 are commonly used.
(from [Oppenheim, Schafer, 1999], v x, V X )
The ow graph in (a) has the disadvantage, that it is a non-inplace algorithm, because the buttery-structure does not continue past the rst stage. Decimation-in-frequency algorithms Instead of applying the decomposition to time domain, we could also start the decomposition in the frequency domain, where the sequence of DFT coefcients VN (k) is decomposed into smaller sequences. The resulting algorithm is called decimationin-frequency (DIF) FFT.
59
Radix-4 buttery (q = 0, . . . , N/4 1) (N N ):

For special lengths, which can not be expressed as N = r m, so called mixed-radix FFT algorithms can be used (i.e. for N = 576 = 26 32).
60
3.3 Transformation of real-valued sequences
Thus, we have
=0
v ( n) I R FFT program/hardware: vR (n) + j vI (n) C I Inefcient due to performing arithmetic calculations with zero values In the following we will discuss methods for the efcient usage of a complex FFT for real-valued data.
3.3.1 DFT of two real sequences Given: v1(n), v2(n) I R, n = 0 , . . . , N 1 How can we obtain VN1 (k) v1(n), VN2 (k) v2(n)? Dene
VN1 (k) =
1 VNR (k) + VNR (N k) + 2 j + VNI (k) VNI (N k) , (3.31) 2
where
VNRe (k) =
v (n) = v1(n) + j v2(n)

leading to the DFT
(3.26)
1 VNR (k) + VNR (N k) , 2 1 VNI (k) VNI (N k) . VNIo (k) = 2
VN (k) = DFT{v (n)} = VN1 (k) +j VN2 (k) .

I C I C
(3.27)
Likewise, we have for VN2 (k) the relation
Separation of VN (k) into VN1 (k), VN2 (k)? Symmetry relations of the DFT:
VN2 (k) =
v (n) = vRe (n) + vRo (n) + j vIe (n) + j vIo (n)

= v1 ( n ) = v2 ( n )
(3.28) with
1 VNI (k) + VNI (N k) 2 j VNR (k) VNR (N k) , (3.32) 2
Corresponding DFTs:
vRe (n) VNRe (k), vRo (n) j VNIo (k), j vIe (n) j VNIe (k), j vIo (n) VNRo (k).
(3.29) (3.30)
61
1 VNI (k) + VNI (N k) , 2 1 VNRo (k) = VNR (k) VNR (N k) . 2 VNIe (k) =
62
Rearranging (3.31) and (3.32) nally yields
Dene
VN1 (k) =
1 VN (k) + VN (N k) , 2
v (n) := v (2n) + j v (2n + 1),

(3.33)
n = 0, . . . , N 1, (3.35)
j VN2 (k) = VN (k) VN (N k) . 2

Due to the Hermitian symmetry of real-valued sequences
=: v1(n) + j v2(n),
where the even and odd samples of v (n) are written alternatively into the real and imaginary part of v (n). Thus we have a complex sequence consisting of two real-valued sequences of length N with the DFT
VN(1,2) (k) = VN
(1,2)
(N k )
(3.34)
the values VN(1,2) (k) for k {N/2 + 1, . . . , N 1} can be obtained from those for k {0, . . . , N/2}, such that only a calculation for N/2+1 values is necessary. Application: Fast convolution of two real-values sequences with the DFT/FFT 3.3.2 DFT of a 2N -point real sequence Given: v (n) I R, n = 0 , . . . , 2 N 1 Wanted:
k = 0, . . . , N 1. (3.36) VN1 (k ) and VN2 (k ) can easily be obtained with (3.33) as VN1 (k ) = 1 (N k ) , VN (k ) + V N 2 j (N k ) VN2 (k ) = VN (k ) V N 2
N (k) = VN (k) + j VN (k), V 2 1
V2N (k) = DFT{v (n)} =
2N 1 n=0
v ( n) W2N
kn
for k = 0, . . . , N 1. In order to calculate V2N (k) from VN1 (k) and VN2 (k) we rearrange the expression for DFT{v (n)} according to
V2N (k) =
N 1
v (2n)
2kn W2N
N 1 n=0
v (2n + 1) W2N
= v2 ( n ) N 1 n=0
(2n+1)k
with k = 0, . . . , 2N 1. Hermitian symmetry analog to (3.34) since v (n) I R for all n:
n=0 = v (n) 1
V2N (2N k) = V2N (k),
N 1 n=0
kn v1 ( n ) W N
k W2N
k = 0, . . . , N
63
v2 ( n ) W N ,
nk
64
Finally we have
4. Digital Filters
k
V2N (k) = VN1 (k)+ W2N VN2 (k), k = 0, . . . , 2N 1

(3.37) Due to the Hermitian symmetry k only needs to be evaluated from 0 to N with VN1/2 (N ) = VN1/2 (0).
V2N (k) = V2 N (2N k ),
Digital lter = linear-time-invariant (LTI) causal system with a rational transfer function (without loss of generality: numerator degree N = denominator degree)
N
Signal ow graph:
N (k ) V k=N k=0 (N k ) V N j 1/2 j
k VN2 (k ) W2 N
1/2
VN1 (k )
V2N (k ) k = 0, . . . , N
H (z ) =
i=0 N i=0
bN i z i aN i zi
bi z i
N
i=0
(4.1)
1+
i=1
ai
z i
Computational savings by a factor of two compared to the complex-valued case since for real-valued input sequences only an N point DFT is needed
where a0 = 1 without loss of generality. ai, bi: parameters of the LTI system ( coefcients of the digital lter), N is said to be the lter order Product notation of (4.1):
N
H (z ) = b0
i=1 N i=1
(z z0i)
(4.2)
(z zi)
where the z0i are the zeros, and the zi are the poles of the transfer function (the latter are responsible for stability). Difference equation:
N N
y ( n) =
i=0
b i v (n i)
i=1
ai y (n i),
(4.3)
65
66
with v (n) denoting the input signal and y (n) the resulting signal after the ltering Remarks
Transfer function:
Generally, (4.3) describes a recursive lter with an innite impulse response (IIR lter): y (n) is calculated from v (n), v (n1), . . . , v (nN ) and recursively from y (n 1), y (n 2), . . . , y (n N ). The calculation of y (n) requires memory elements in order to store v (n 1), . . . , v (n N ) and y (n 1), y (n 2), . . . , y (n N ) dynamical system. bi 0 for all i = 0: H (z ) = b0 z N
N i=0
1 H (z ) = N z
N i=0
bN i z =
bi z
i=0
(4.7)
b0 z N
N i=1
Poles zi = 0, i = 1, . . . , N , but not relevant for stability all-zero lter According to (4.6): y (n) obtained by a weighted average of the last N + 1 input values Moving average (MA) lter (as opposite to the AR lter from above) From (4.7) it can be seen that the impulse response has nite length Finite impulse response (FIR) lter of length L = N + 1 and order N 4.1 Structures for FIR systems
(4.4)
aN i z i
(z zi)
Filter has no zeros All-pole or autoregressive (AR-) lter. Transfer function is purely recursive:
N
Difference equation given by (4.6) Transfer function given by (4.7) bn 0 Unit sample response is equal to the coefcients bi h(n) =
for otherwise
0 n L1
y ( n) = b 0 v ( n)
i=1
ai y (n i)
(4.5)
4.1.1 Direct form structures
ai 0 for all i = 0, a0 = 1 (causal lter required!): Difference equation is purely non-recursive:

N
The direct form structure follows immediately from the nonrecursive difference equation given in (4.6), which is equivalent to the linear convolution sum
y ( n) =
i=0
b i v (n i)
(4.6)
y ( n) =
L1 k=0
Non-recursive lter
67
h(k) v (n k).
68
Transversal lter in the second direct form:

v (n)
h(L 1) z 1
h(L 2) z 1
h(3) z 1
h(2) z 1
h(1) z 1
h(0)
y (n)
(from [Proakis, Manolakis, 1996], v x, L M )
Tapped-delay-line or transversal lter in the rst direct form

If the unit impulse v (n) = (n) is chosen as input signal, all samples of the impulse response h(n) appear successively at the output of the structure. In the following we mainly use the more compact signal ow graph notation:
v (n) z 1 z 1 z 1 z 1 z 1 z 1
When the FIR lter has linear phase (see below) the impulse response of the system satises either the symmetry or asymmetry condition h(n) = h(L 1 n). (4.8) Thus, the number of multiplications can be reduced from L to L/2 for L even, and from L to (L +1)/2 for L odd. Signal ow graph for odd L:
z 1 z 1 z 1 z 1 z 1 z 1 z 1 v (n) z 1 z 1 z 1 z 1 z 1 z 1
h(0)
h(1)
h(2)
h(3)
h(L 2)
h(L 1)
h(0) y (n)
h(1)
h(2)
h(3)
L3 2
L1 2
y (n)
The second direct form can be obtained by transposing the ow graph:
4.1.2 Cascade-form structures By factorizing the transfer function

P
Reversing the direction of all branches, exchanging the input and output of the ow graph exchanging summation points with branching points and vice versa.
69
H (z ) = H0
p=1
Hp(z )
(4.9)
we obtain a cascade realization. The Hp(z ) are normally secondorder, since in order to obtain real coefcients, conjugate complex
70
zeros z0i and z0 i have to be grouped:
The behavior of the i-th stage can be written in matrix notation as

1
Hp(z ) = (1 z0iz = 1 + b1 z
)(1 z0iz
2
+ b2 z
Xi(z ) 1 = Yi(z ) qi
qi z 1 Xi1(z ) . z 1 Yi1(z )
(4.10)
After the rst stage we have For linear-phase lters due to the special symmetry (4.8) the zeros 1 1 are appear in quadruples: Both z0i and z0 i , and z0i and (z0i ) a pair of complex-conjugate zeros. Consequently, we have
X1(z ) = V (z ) + q1 z Y1(z ) = q1 V (z ) + z z ),
It follows
1 1
V (z ),
(4.11)
V (z ).
Hp(z ) = (1 z0iz = 1 + b1 z
)(1 z0iz + b2 z
2
)(1 z0i z
3
1 1 4
)(1 (z0i) .
1 1
+ b1 z
+z
H1(z ) =
4.1.3 Lattice structures Lattice structures are mainly used as predictor lter (i.e. in digital speech processing) due to their robustness against coefcient quantization:
General structure:
x1 (n) v (n) 1-th stage y1 (n) x2 (n) 2-th stage y2 (n) (L 1)-th stage xL1 (n) yL1 (n)
X1(z ) 1 1 = 1 + q1 z = 01 + 11 z , V (z ) Y1(z ) 1 1 = q1 + z = 01 + 11 z . V (z )
G1 ( z ) =
Second stage:
X2(z ) =X1(z ) + q2 z Y2(z ) =q2 X1(z ) + z

Inserting (4.11) into (4.12) yields
1 1
Y1(z ), Y1(z ).
(4.12)
i-th stage lattice lter:

xi1 (n) 1 1 qi z 1 1 qi 1 1 xi (n)
X2(z ) = V (z ) + q1z
yi (n)
V (z ) + q2q1z
1
V (z ) + q2z V (z ) + z
V (z ),
yi1 (n)
Y2(z ) = q2V (z ) + q1q2z

71
V (z ) + q1z
V (z ),
72
which nally leads to the transfer functions
we obtain the direct form I realization:

v (n) z 1 b0 z 1 b1 a1 z 1 b2 z 1 b3 z 1 a3 z 1 a2 z 1 y (n)
H2(z ) =
X2(z ) 1 2 = 1 + (q1 + q1 q2)z + q2 z , (4.13) V (z )

1
= 02 + 12 z G2 ( z ) =
+ 22z
,
(4.14)
z 1
Y2(z ) 1 2 = q2 + (q1 + q1 q2)z + z , V (z )

1
= 02 + 12 z
+ 22z
By comparing (4.13) and (4.14) we can see that
H2(z ) = z
G2 ( z
),
that is, the zeros of H2(z ) can be obtained by reecting the zeros of G2(z ) at the unit circle. Generally, it can be shown that
z 1 b N 1 z 1 bN aN aN 1
z 1
Hi(z ) = z
Gi ( z
),
for
i = 1, . . . , L 1.
(4.15)
z 1
4.2 Structures for IIR systems 4.2.1 Direct form structures Rational system function (4.1) can be viewed as two systems in cascade: H (z ) = N (z )/D (z ) = H1(z ) H2(z ) with
N
All-zero system N (z )
All-pole system 1/D(z )
Another realization can be obtained by exchanging the order of the all-pole and all-zero lter. Then, the difference equation for the all-pole section is
N
H1(z ) =
i=0
bi z ,
H2(z ) = 1+
1
N i=1
ai z i
w ( n) =
i=1
ai w(n i) + v (n),
The all-zero H1(z ) can be realized with the direct form from Section 3.1.1. By attaching the all-pole system H2(z ) in cascade,
73
where the sequence w(n) is an intermediate result and represents

74
the input to the all-zero section:

N
structure, which requires the same number of multiplications, additions, and memory locations as the original structure:
y ( n) =
i=0
b n w (n i).
v (n)
b0 z b1 z 1 b2 a2 z 1 b3 a3 z 1
1
y (n)
The resulting structure is given as follows:

v (n) z 1 a1 z 1 a2 z 1 a3 z 1 b3 b2 b1 b0 y (n)
a1
z 1 b N 1 z 1 bN aN aN 1
z 1 aN 1 z 1 aN bN b N 1
4.2.2 Cascade-form structures Analog to Section 4.1.2 we can also factor an IIR system H (z ) into rst and second order subsystems Hp(z ) according to
P
Only one single delay line is required for storing the delayed versions of the sequence w(n). The resulting structure is called a direct form II realization. Furthermore, it is said to be canonic, since it minimizes the number of memory locations (among other structures). Transposing the direct form II realization leads to the following
75
H (z ) =
p=1
Hp(z ).
(4.16)
Degrees of freedom in grouping the poles and the zeros
76
First order subsystems: Canonical direct form for N = 1:

v (n) z a1
1
Example:
A so called Chebyshev lowpass lter of 5-th order and the cut-off frequency fco = 0.25 fs (fs denoting the sampling frequency) is realized. A lter design approach (we will discuss the corresponding algorithms later on) yields the transfer function
b0
y (n)
b1
H (z ) = 0.03217
Corresponding transfer function:
H (z ) =
Y (z ) b0 + b1 z 1 = V (z ) 1 + a1 z 1
(4.17)
1 + 5 z 1 + 10 z 2 + 10 z 3 + 5 z 4 + z 5 1 0.782 z 1 + 1.2872 z 2 0.7822 z 3 + 0.4297 z 4 0.1234 z 5 (4.19) The zeros are all at z = 1: z0i = 1 for i = 1, 2, . . . , 5. The location of the poles are: z1,2 = 0.0336 j 0.8821, z3,4 = 0.219 j 0.5804,
Grouping of poles z1,2 : (4.20)
Every rst order transfer function can be realized with the above ow graph:
1 1 (b b0 + b1 z 1 b 0 /a0 ) + (b1 /a0 ) z 0 + b1 z = = H (z ) = 1 1 a0 + a 1 + (a 1 + a1 z 1 1z 1 /a0 ) z
z5 = 0.4113.
Second order subsystems: Canonical direct form for N = 2:

v (n) z 1 a1 z 1 a2 b2 b1 b0 y (n)
1,2 (z ) = H
1 + 2 z 1 + z 2 1 + 0.0672 z 1 + 0.7793 z 2
Grouping of poles z3,4 :
3,4 (z ) = H
1 + 2 z 1 + z 2 1 0.4379 z 1 + 0.3849 z 2
Real-valued pole z5 leads to a rst-order subsystem
Corresponding transfer function:
5 (z ) = H
1 + z 1 . 1 0.4113 z 1
H (z ) =
b0 + b1 z 1 + b2 z 2 Y (z ) = V (z ) 1 + a1 z 1 + a2 z 2
(4.18)
For the implementation on a xed-point DSP it is advantageous to ensure that all stages have similar amplication in order to avoid numerical problems. Therefore, all subsystems are scaled such that they have
78
77
approximately the same amplication for low frequencies:
Resulting signal ow graph (V U ):
H1 (z ) = H2 (z ) =
5 (z ) H 5 (z = 1) H =
0.2943 + 0.2943 z 1 1 0.4113 z 1
3,4 (z ) H 3,4 (z = 1) H 1,2 (z ) H 1,2 (z = 1) H
0.2367 + 0.4735 z 1 + 0.2367 z 2 1 0.4379 z 1 + 0.3849 z 2 0.4616 + 0.9233 z + 0.4616 z 1 0.4379 z 1 + 0.3849 z 2
1 2
(from [Fliege: Analoge und digitale Filter, Hamburg University of Technology, 1990])
H3 (z ) =
4.2.3 Parallel-form structures
Remark: The order of the subsystems is in principle arbitrary. However, 1,2 (z ) here we know from the pole analysis in (4.20) that the poles of H are closest to the unit circle. Thus, using a xed-point DSP may lead more 3,4 (z ) and H 5 (z ). Therefore, likely to numerical overow compared to H it is advisable to realize the most sensible lter as the last subsystem. Frequency responses:
20 log 20 0 dB dB 20 40 0 0.5 / 20 log 20 0 dB 20 40 0 0.5 / 1 dB
10 10
An alternative to the factorization of a general transfer function is to use a partial-fraction expansion, which leads to a parallelform structure In the following we assume that we have only distinct poles (which is quite well satised in practice). The partial-fraction expansion of a transfer function H (z ) with numerator and denominator degree N is then given as
N
|H (e )|
1
20 0 20 40 1 0
20 log10 |H2(e )|
H ( z ) = A0 +
i=1
Ai . 1 ziz 1
(4.21)
0.5 /
10
|H (ej )|
3
20 log 20 0 20 40 0
|H(ej )| (overall filter)
The Ai, i {1, . . . , N } are the coefcients (residues) in the partial-fraction expansion, A0 = bN /aN . We furthermore assume that we have only real-valued coefcients, such that we can combine pairs of complexconjugate poles to form a second order subsystem (i {1, . . . , N }):
A Ai i + z 1 = 1 zi z 1 1 z i
0.5 /
1 2 {Ai } 2 {Ai z b0 + b1 z 1 i }z =: 1 2 {zi }z 1 + |zi |2 z 2 1 + a1 z 1 + a2 z 2 (4.22)
79
80
Two real-valued poles can also be combined to a second order transfer function (i, j {1, . . . , N }):
Ai + = 1 zi z 1 1 zj z 1 Aj
Example:
Consider again the 5-th order Chebyshev lowpass lter with the transfer function from (4.19). The partial fraction expansion can be given as
1 (zj + zi ) z 1 + (zj zi ) z 2
(Ai + Aj ) (Ai zj + Aj zi ) z
=:
b0 + b1 z 1 + a1 z 1 + a2 z 2 (4.23)
H (z ) = 0.2607 + +
If N is odd, there is one real-valued pole left, which leads to one rst order partial fraction (see example).
Parallel structure:

A A5 A3 3 + + 1 1 1 z3 z 1 z3 z 1 z5 z 1 A1 = 0.1 + j 0.0941,
A1 A 1 + z 1 + 1 z1 z 1 1 z 1
with the poles from (4.20) and the residues
z1 = 0.0336 + j 0.8821, z3 = 0.219 + j 0.5804, z5 = 0.4114,
A3 = 0.5533 + j 0.00926, A5 = 1.1996.
With (4.22) the resulting transfer function writes
H (z ) = 0.2607 +
0.2 0.1592 z 1 + 1 + 0.0673 z 1 + 0.7793 z 2
: number of parallel subsystems
1.1996 1.1066 + 0.3498 z 1 + . 1 0.4379 z 1 + 0.3849 z 2 1 0.4114 z 1

Resulting signal ow graph (V U ):
Signal ow graph of a second order section:

v (n) z ap1 z 1 ap2 p = 1, . . . , P
1
bp0
yp (n)
bp1
(from [Fliege: Analoge und digitale Filter, Hamburg University of Technology, 1990])
81
82
can be represented with one of the three following formats 4.3 Coefcient quantization and round-off effects In this section we discuss the effects of a xed-point digital lter implementation on the system performance. 4.3.1 Errors resulting from rounding and truncation Number representation in xed-point format: A real number v can be represented as
B
Sign-magnitude format: vSM = [1.12 . . . b1] for v < 0. Ones-complement format:

(4.27)
v = [A, . . . , 1, 0, 1, . . . , B ] =
r
=A
1 2 . . . b1] for v < 0, v1C = [1.
(4.28)
(4.24) where is the digit, r is the radix (base), A the number of integer digits, B the number of fractional digits. Example: [101.01]2 = 1 22 + 0 21 + 1 20 + 0 21 + 1 22 Most important in digital signal processing:
= 1 denoting the ones complement of . with

Alternative denition:
0 b1 =1 b+1
v1 C = 1 2 +
(1 ) 2
= 22
|v | (4.29)
Positive numbers are represented as
Binary representation with r = 2 and {0, 1}, A MSB, B LSB. b-bit fraction format: A = 0, B = b1, binary point between 0 and 1 numbers between 0 and 2 2b+1 are possible.
b1 =1
Twos-complement format: 1 2 . . . b1] + [00 . . . 01] for v < 0, v2C = [1. (4.30) where + denotes a binary addition. We thus have by using (4.29) v2 C = v1 C + 2
b+1
v = [0.12 . . . b1] =
Negative fraction:
2 .
(4.25)
= 2 |v | .
(4.31)
Does (4.30) really represent a negative number? Using the identity

b1 =1
v = [0.12 . . . b1] =
2 ,
(4.26)
1=
b1 =1
+2
b+1
83
84
we can express a negative number as
v=
b1 =1
2
b1 =1
+11
Important property: If the sum of numbers is within the range it will be computed correctly, even if individual partial sums result in overow.
Truncation and rounding:
= 1 +
Example:
(1 ) 2
=
+2
b+1
= v2 C 2 .
Express the fractions 7/8 and 7/8 in sign-magnitude, twos-complement, and ones-complement format.
Problem: Multiplication of two b-bit numbers yields a result of length (2b1) truncation/rounding necessary can again be regarded as quantization of the (lter) coefcient v Suppose that we have a xed-point realization in which a number v is quantized from bu to b bits. We rst discuss the truncation case. Let the truncation error be dened as Et = Qt[v ] v .
v = 7/8 can be represented as 21 + 22 + 23 , such that v = [0.111]. In sign-magnitude format, v = 7/8 is represented as vSM = [1.111], in ones complement we have v1C = [1.000], and in twos-complement the result is v2C = [1.000] + [0.001] = [1.001].
(For further examples see also the table in Section 2.4.)
For positive numbers the error is (2

b+1
Remarks: Most DSPs use twos-complement arithmetic. Thus any b-bit number v has the number range
bu +1
) Et 0
(truncation leads to a number smaller than the unquantized number)
v {1, 1 + 2
b+1
,...,1 2
b+1
}.
For negative numbers and the sign-magnitude representation the error is 0 Et (2

b+1
Twos-complement arithmetic with b bits can be viewed as arithmetic modulo 2b (Example for b = 3):
bu +1
).
(truncation reduces the magnitude of the number)
For negative numbers in the twos-complement case the error is b+1 b +1 (2 2 u ) Et 0 (negative of a number is obtained by subtracting the corresponding positive number from 2, see (4.31))
85 86
Quantization characteristic functions for a continuous input signal v (v x):

Sign-magnitude representation: Twos-complement representation:
4.3.2 Numerical overow If a number is larger/smaller than the maximal/minimal possible number representation
+1
+1
(1 2b+1) for sign magnitude and oness-complement arithmetic;

+1
+1
1 and 1 2b+1, resp., for twos-complement arithmetic,

we speak of an overow/underow condition. Overow example in twos-complement arithmetic (range: 8, . . . , 7)
Rounding case, rounding error is dened as Er = Qr [v ] v :
[0.111] + [0.001] = [1.000]

7 1 8
Rounding affects only the magnitude of the number and is thus independent from the type of xed-point realization. Rounding error is symmetric around zero and falls in the range 1 b+1 1 b+1 b +1 b +1 2 u ) Er (2 2 u ). (2 2 2 Quantization characteristic function, bu = (v x):
resulting error can be very large when overow/underow occurs Twos-complement quantizer for b = 3, = 2b (v x):
+1
+1
Alternative: saturation or clipping, error does not increase abruptly in magnitude when overow/underow occurs:
87 88
As an example, we are interested in the deviation zi = zi z i, when the denominator coefcients ai are quantized (z i denotes the resulting pole after quantization). It can be shown [Proakis, Manolakis, 1996, pp. 569] that this deviation can be expressed as:
N N n z i
zi =
n=1
N =1, =i
an,
i = 1, . . . , N.
(4.32)
(zi z)
Disadvantage: Summation property of the twos-complement representation is violated 4.3.3 Coefcient quantization errors
From (4.32) we can observe the following:
By using the direct form, each single pole deviation zi depends on all quantized denominator coefcients a i. The error zi can be minimized by maximizing the distance |zi z| between the poles zi and z. Splitting the lter into single or double pole sections (rst or second order transfer functions):
Combining the poles zi and z i into a second order section leads to a small perturbation error zi, since complex conjugate poles are normally sufciently far apart.
In a DSP/hardware realization of an FIR/IIR lter the accuracy is limited by the wordlength of the computer Coefcients obtained from a design algorithm have to be quantized Wordlength reduction of the coefcients leads to different poles and zeros compared to the desired ones. This may lead to modied frequency response with decreased selectivity stability problems
Sensitivity to quantization of lter coefcients Direct form realization, quantized coefcients:
a i = ai + ai, bi = bi + bi ,
i = 1, . . . , N, i = 0, . . . , N,
Realization in cascade or parallel form: The error of a particular pole pair zi and z i is independent of its distance from the other poles of the transfer function.
Example: Effects of coefcient quantization
Elliptic lter of order N = 12 (Example taken from [Oppenheim, Schafer, 1999]):
89 90
ai and bi represent the quantization errors.
Poles: z1,2 = r ej , Coefcients: a1 = 2r cos , a2 = r 2, Stability condition: |r | 1
v (n) z 1 2r cos z 1 r2
y (n)
Quantization of a1 and a2 with b = 4 bits: possible pole positions:
Unquantized: (a) Magnitude frequency response 20 log10 |H (ej )| (b) Passband details Quantized with b = 16 bits: (c) Passband details for cascade structure (d) Passband details for parallel structure (e) Magnitude frequency response (log) for direct structure
Nonuniformity of the pole position is due to the fact that a2 = r 2 is quantized, while the pole locations z1,2 = r ej are proportional r . Sparse set of possible pole locations around = 0 and = . Disadvantage for realizing lowpass lters where the poles are normally clustered near = 0 and = .
Alternative: Coupled-form realization
Pole locations of quantized second order sections Consider a two-pole lter with the transfer function
y1(n) = v (n) + r cos y1(n 1) r sin y (n 1), y (n) = r sin y1(n 1) + r cos y (n 1),
(4.33)
92
H (z ) =
1 . 1 (2r cos ) z 1 + r 2 z 2
91
which corresponds to the following signal ow graph:

v (n) r cos r sin r cos r sin y (n) z 1 y1 (n) z 1
Disadvantage: Increased computational complexity compared to the direct form.

Cascade or parallel form?
Cascade form: By transforming (4.33) into the z-domain, the transfer function of the lter can be obtained as
H (z ) =
bp0 + bp1 z 1 + bp2 z 2 1 + ap1 z 1 + ap2 z 2 p=1

P
Parallel form:
H ( z ) = A0 +
p=1
cp0 + cp1 z 1 1 + ap1 z 1 + ap2 z 2
Y (z ) (r sin ) z 1 H (z ) = = . V (z ) 1 (2r cos ) z 1 + r 2 z 2 We can see from the signal ow graph that the two coefcients r sin and r cos are now linear in r , such that a quantization of these parameters lead to equally spaced pole locations in the z-plane:
Cascade form: Only the numerator coefcients bpi of an individual section determine the perturbation of the corresponding zero locations (an equation similar to (4.32) can be derived) direct control over the poles and zeros Parallel form: A particular zero is affected by quantization errors in the numerator and denominator coefcients of all individual sections numerator coefcients cp0 and cp1 do not specify the position of a zero directly, direct control over the poles only Cascade structure is more robust against coefcient quantization and should be used in most cases
Example:
Elliptic lter of order N = 7, frequency and phase response (taken from [Proakis, Manolakis, 1996]):
93
94
Cascade form (3 digits = b 10 bits, 4 digits = b 14 bits)
Coefcient quantization in FIR systems In FIR systems we only have to deal with the locations of the zeros, since for causal lters all poles are at z = 0. Remarks:
For FIR lters an expression analogous to (4.32) can be derived for the zeros FIR lters should also be realized in cascade form according to
P
H (z ) = H0
p=1
(1 + bp1 z
+ b p2 z
Parallel form (b = 10 bits)
with second order subsections, in order to limit the effects of coefcient quantization to the zeros of the actual subsection only.
However, since the zeros are more or less uniformly spread in the z-plane, in many cases the direct form is also used with quantized coefcients. For a linear-phase lter with the symmetry (4.8) in the impulse response, quantization does not affect the phase characteristics, but only the magnitude.
4.3.4 Round-off effects Direct-form I IIR structure Starting point for the analysis: Direct-form I structure with the difference equation
N N
y ( n) =
i=0
b i v (n i)
i=1
ai y (n i).
96
95
All signal values and coefcients are represented by b-bit binary xed-point numbers (for example in twos-complement representation):
As already stated in Section 2.4 the result of each single quantization stage can be modeled by adding a noise source ei(n) with the following properties:
truncation or rounding of the (2b 1)-bit products to b bit necessary modelling as a constant multiplication followed by a quantizer
Each ei(n) corresponds to a wide-sense-stationary whitenoise process. Each ei(n) has an uniform distribution of amplitudes over one quantization interval (uniform p.d.f.). Each ei(n) is uncorrelated with the quantizer input, all other quantization noise sources and the input signal of the lter.
We have shown above that for b-bit quantization the rounding error falls in the range
1 b+1 1 b+1 (2 ) ei(n) (2 ), 2 2

and for twos-complement truncation we have
b+1
ei ( n ) 0 .
Mean and variance for rounding
e = 0,
and for truncation
e =
22b+2 , 12
(4.35)
(from [Oppenheim, Schafer, 1999], v x)
This can be described with the difference equation

N N
e =
2b+1 , 2
e =
22b+2 . 12
(4.36)
Autocorrelation (white noise process):
y ( n) =
i=0
Q[bi v (n i)]
i=1
Q[ai y (n i)]. (4.34)

97
ee(n) = e (n) + e .
(4.37)
98
In the following we will restrict ourselves to the rounding case, 2 where ee(n) = e (n) and thus, for the power spectral density j 2 we have (e ) = e :
quantization noise e(n) now is given as

N
The following structure can be obtained by summing up all the noise sources:
4
f ( n) =
i=1
aif (n i) + e(n),
(4.39)
e( n ) =
i=0
ei ( n ) ,
since e(n) can be regarded as the input to an all-pole system with output f (n).
Suppose that the allpole-lter has the transfer function Hef (z ) with
Hef (z ) =
1 , D (z )
H (z ) =
N (z ) D (z )
analog to (4.1).
Mean of f (n): f = e Hef (ej 0) = 0 for rounding.
Overall noise variance in the special case from above:

4 2 e
=
i=0
e = 5
2 i
22b+2 12
2 1 Variance f = F {f f (ej )}|n=0: 2 e 2
Power spectral density (power spectrum) 2 f f (ej ) = e |Hef (ej )|2.
Overall noise variance in the general case:

2 e
f =
(4.38)
|Hef (e )| d = e
n=
|hef (n)| ,
= (2N + 1)
2b+2
12
(4.40) where the last expression is obtained by applying Parsevals theorem. By combining (4.38) and (4.40) we can now state the total output
100
Due to linearity the output of the whole lter is y (n) = y ( n) + f ( n) . Thus, the difference equation for the
99
variance due to internal round-off as
approaches the unit circle In order to neep the noise variance below a certain level, the wordlength b has to be increased.
2 f
22b+2 = (2N +1) 2 12
d |D (ej )|2
Round-off noise in a second-order system (4.41) Second-order direct form I system:
22b+2 2 = (2N +1) |hef (n)| 12 n=

Round-off noise in a rst-order system Given: Transfer function
2 f
H (z ) =
Thus we have
b0 + b1 z 1 + b2 z 2 (1 r ej z 1)(1 r ej z 1)
H (z ) =
b0 , 1 a1z 1
22b+2 1 =5 12 2
|a1| < 1
d . |(1 r ej ej )|2 |(1 r ej ej )|2
Impulse response: h(n) = b0 an 1 u(n) (u(n): unit step function)

e(n) = ea1 (n) + eb0 (n)
With a1 = 2r cos and a2 = r 2 it can be shown via a partial fraction expansion that
f = 5
22b+2 12
1 + r2 1 r2
1 . (4.43) r 4 + 1 2r 2 cos(2 )
v (n)
b0 z 1 a1
y (n)
As in the rst-order case we can see that the total variance increases if the poles approach the unit circle (r 1). Direct-form II structure In this case, the nonlinear difference equations are of the form
N
Considering (4.41) and the two error sources ea1 (n), eb0 (n), we have
2 f
22b+2 =2 12
n=0
|a1|
2n
22b+2 = 6
(4.42) The output noise variance increases when the pole z = a1

101
1 1 |a1|2
w ( n) =
N
i=1
Q[ai w(n i)] + v (n),

(4.44)
y ( n) =
i=0
Q[bi w(n i)].

102
Signal ow graph:
and by applying a relation similar to (4.40)

2 f
22b+2 =N 12
n=
|h(n)| + (N + 1)
22b+2 . 12
(4.47)
White noise produced in implementing the poles is ltered by the entire system, whereas the white noise produced in implementing the zeros is added directly to the output of the system. A comparison with (4.41) for the direct form I structure shows that both structures are affected differently by the quantization of products.
4.3.5 Zero-input limit cycles
For rounding (e = 0) the power spectrum of the output noise is:
f f (e ) = N
We then have:
22b+2 22b+2 j 2 |H (e )| + (N + 1) . 12 12 (4.45)
Stable IIR lters implemented with innite-precision arithmetic: If the excitation becomes zero and remains zero for n > n0 then the output of the lter will decay asymptotically towards zero. Same system implemented with xed-point arithmetic: Output may oscillate indenitely with a periodic pattern while the input remains equal to zero: Zero-input limit cycle behavior, due to nonlinear quantizers in the feedback loop or overow of additions.
No general treatment, but two examples are given Limit cycles due to round-off and truncation Given: First-order system with the difference equation
2 f
22b+2 1 =N 12 2
22b+2 , |H (e )| d + (N + 1) 12
j 2
(4.46)
103
y (n) = a y (n 1) + v (n),
|a| < 1.
104
Register length for storing a and the intermediate results: 4 bits (sign bit plus 3 fractional digits) product a y (n 1) must be rounded or truncated to 4 bits, before being added to v (n) Signal ow graphs:
Innite-precision linear system v (n) z a
1
A constant steady value is obtained for n 3.
For a = 1/2 we have a periodic steady-state oscillation between 1/8 and 1/8.
Such periodic outputs are called limit cycles.
y (n)
Nonlinear system due to quantization v (n) Q[] a z 1 y (n)
Nonlinear difference equation (Q[] represents twos-complement rounding):
y (n) = Q[a y (n 1)] + v (n).

Suppose we have a = 1/2 = [0.100], v (n) = 7/8 (n) = [0.111] (n):
y (0) = 7/8 = [0.111] y (1) = Q[a y (0)] = Q [ [0.100] [0.111] ] = Q [ [0.011100] ] = Q[7/16] = [0.100] = 1/2 y (2) = Q[a y (1)] = [0.010] = 1/4 y (3) = Q[a y (2)] = [0.001] = 1/8 y (4) = Q[a y (3)] = Q [ [0.000100] ] = [0.001] = 1/8
105
Limit cycles due to overow Consider a second-order system realized by the difference equation
y (n) = v (n) + Q [a1 y (n 1)] + Q [a2 y (n 2)] , (4.48) Q[] represents twos-complement rounding with one sign and 3 fractional digits.
106
Overow can occur with the twos-complement addition of the products in (4.48). Suppose that a1 = 3/4 = [0.110], a2 = 3/4 = [1.010], y (1) = 3/4 = [0.110], y (2) = 3/4 = [1.010], v (n) = 0 for all n 0:
4.4 Design of FIR lters General remarks (IIR and FIR lters)
Ideal lters are noncausal, and thus physically unrealizable for real-time signal processing applications. Causality implies that the a lter frequency response H (ej ) cannot have an innitely sharp cutoff from passband to stopband, and that the stopband amplication can only be zero for a nite number of frequencies .
Magnitude characteristics of physically realizable lters:
y (0) = Q[ [0.110] [0.110] ] + Q[ [1.010] [1.010] ] = Q[ [0.100100] ] + Q[ [0.100100] ] = [0.101] + [0.101] = [1.010] = 3/4 y (1) = [1.011] + [1.011] = [0.110] = 3/4 y (n) continues to oscillate unless an input is applied.
Remarks Some solutions for avoiding limit cycles:
Use of structures which do not support limit-cycle oscillations. Increasing the computational wordlength. Use of a double-length accumulator and quantization after the accumulation of products.
FIR-lters are limit-cycle free since there is no feedback involved in its signal ow graph.
1: passband ripple, 2: stopband ripple, p: passband edge frequency, s: stopband edge frequency
Filter design problem:
Specify 1, 2, p, and s corresponding to the desired application. Select the coefcients ai and bi (free parameters), such that the resulting frequency response H (ej ) best satises the requirements for 1, 2, p, and s. The degree to which H (ej ) approximates the specications depends on the criterion used for selecting the ai and bi and
107
108
also on the numerator and denominator degree N (number of coefcients) 4.4.1 Linear phase lters Important class of FIR lters, which we will mainly consider in the following. Denition: A lter is said to be a linear-phase lter, if its impulse response satises the condition (L = N +1):
which is the z-transform equivalent to (4.49). Consequences:
The roots of the polynomial H (z ) are identical to the roots of 1 the polynomial H (z 1): If z0i is a zero of H (z ) then z0 i is also a zero. If additionally the impulse response h(n) is real-valued, the roots must occur in complex-conjugate pairs: If z0i is a zero of H (z ) then z0 i is also a zero. The zeros of a real-valued linear-phase lter occur in quadruples in the z-plane (exception: zeros on the real axis, zeros on the unit circle)
h(n) = h(L 1 n).
(4.49)
With the denition S := (L 1)/2 and for L odd this leads to a z-transform
H (z ) =
L1 n=0
h(n)z
(4.50)
S 1 n=0
=z
h(S ) +
h(n) z
(S n)
(S n)
,
(4.51)
for L even we have
(a) Type-1 linear phase system
H (z ) = z
L/21 n=0
h(n) z
(S n)
(S n)
(4.52)
Denition: Odd length L, even symmetry h(n) = h(L 1 n) Frequency response from (4.51):
When we now substitute z 1 for z in (4.50) and multiply both sides by z (L1) we obtain with (4.49)
H (e ) = e
jS
h(S ) + 2
S 1 n=0
h(n) cos ((S n) )

(4.54)
(L1)
H (z
) = H (z ),
(4.53)
109
=e
jS
H01( )
110
H01( ): amplitude frequency response, real-valued (generally denoted with H0( )) linear phase 1( ) = arg H (ej ) = S
h(n) type 1, odd length (L=31) 0.5
type 3, odd length (L=31)
h(n)
0.5 0 0 1 H ( ) 0 5 15 20 n type 1, amplitude frequency response 10 25 30 0.5

03
10 15 20 n type 3, amplitude frequency response
25
30
0 0.5
1 1 H ( ) 0 0.5 0.5 1 1.5

01
2 /
2.5
3.5
0 0 0.5 1 1.5 2 / 2.5 3 3.5 4
(c) Type-2 linear phase system Denition: Even length L, even symmetry h(n) = h(L 1 n) Frequency response from (4.52):
(b) Type-3 linear phase system Denition: Odd length L, odd symmetry h(n) = h(L1n) Frequency response from (4.51):
H (e ) = e
jS
j h(S ) + 2
S 1 n=0
h(n) sin ((S n) )

(4.55)
H (e ) = e =e
jS
jS
L/21 n=0
H02( )
h(n) cos ((S n) ) (4.56)
=e
jS
j H03( ) = e
jS +j/2
H03( ) linear phase 2( ) = arg H (ej ) = S S = (2 1)/2, I N H (ej ) = 0

111 112
linear phase 3( ) = arg H (ej ) = S /2 H ( ej 0 ) = 0 , S I N H (ej ) = 0
Amplitude frequency response has 4 -periodicity: H02( ) = 2

L/21 n=0 L/21 n=0 L/21 n=0
(d) Type-4 linear phase system Denition: Even length L, odd symmetry h(n) = h(L1n) Frequency response from (4.52):
h(n) cos ((S n) ) h(n) cos ((S n)( + 2 )) h(n) cos ((S n) ) cos ((S n)2 )
H02( + 2 ) = 2
H (e ) = e
jS
j 2
L/21 n=0
h(n) sin ((S n) )

jS +j/2
(4.57)
=2
=e
jS
j H04( ) = e
H04( )
= cos((L12n) ) = 1
= H02( ) Remark: H (ej ) is 2 -periodic again due to the ejS phase factor
type 2, even length (L=30)
linear phase 4( ) = arg H (ej ) = S /2 H ( ej 0 ) = 0 Similar to the type-2 lter the amplitude frequency response has 4 -periodicity: H04( + 2 ) = H04( ). The multiplication with the exponential factor ejS+j/2 then again leads to 2 -periodicity for H (ej ).
type 4, even length (L=30) 0.5
h(n)
0
h(n)
0.5 0
0 5 15 20 n type 2, amplitude frequency response 10 25 30
10 15 20 n type 4, amplitude frequency response
25
30
1 0.5 H ( )
04
0.5 H02( ) 0
0.5 1 0 0.5 1 1.5 2 / 2.5 3 3.5 4
0.5 1 0 0.5 1 1.5 2 / 2.5 3 3.5 4
113
114
Applications:
L with the desired frequency response Hd(e ) =

j
Type-1 and Type-2 lters used for ordinary ltering, however type 2 lters are unsuitable for highpass ltering Type-3 and Type-4 lters for example used for 90 degree phase shifters and so called Hilbert transformers
4.4.2 Design of linear-phase lters using a window function Given: Desired frequency response
ej(L1)/2 0
for
otherwise,
0 | | < c,
(4.60)
where c is denoting the cut-off frequency. Corresponding unit sample response:
Hd(e ) =
n=
hd(n) e
jn
1 hd(n) = 2
(4.58)
e
c
j (n(L1)/2)
d. L1 2
sin c n n
Impulse response hd(n) can be obtained with the inverse Fourier-Transform 1 hd(n) = 2
L1 2 L1 2
n=
(4.61)
Multiplication with a rectangular window of length L leads to

h(n) =
1 sin c n L 2
Hd(e ) e
jn
d.
(4.59)
For L odd:
1 n L 2
n=
L1 , 2
n = 0, . . . , L 1
Impulse response has generally innite length truncation to length L by multiplication with a window function w(n) necessary: h(n) = hd(n) w(n). Rectangular window:
w(n) = 1 n = 0, . . . , L 1 h(n) = 0 otherwise hd (n) n = 0, . . . , L 1 otherwise 0
L1 2
c
1.2 1 |H(ej )|
Example for c = 0.2 , L = 61 and L = 101:

1.2 1 0.8 0.6 0.4 0.2 L=61, c=0.2
|H(ej )|
0.8 0.6 0.4 0.2
L=101, c=0.2
Frequency response of the rectangular window: see (3.12) Suppose, we want to design a linear-phase lowpass lter of length
115
0 0
0.2
0.4 0.6 /
0.8
0 0
0.2
0.4 0.6 /
0.8
Disadvantage of using an rectangular window:

116
Large sidelobes lead to in undesirable ringing effects (overshoot at the boundary between pass- and stopband) in the frequency response of the resulting FIR lter
(n n, L M , I0 : Bessel function of the rst kind of order zero)
Gibbs phenomenon:
Result of approximating a discontinuity in the frequency response with a nite number of lter coefcients and a mean square error criterion Eq. (4.58) can be interpreted as a Fourier series representation for Hd(ej ) with the Fourier coefcients hd(n) Gibbs phenomenon resulting from a Fourier series approximation
Squared integral error E =
approaches zero with increasing length of h(n). However, the maximum value of the error |Hd(ej )H (ej )| approaches a constant value.
(Hd(ej ) H (ej ))2 d
Use of other appropriate window functions with lower sidelobes in their frequency responses
Frequency-domain characteristics of some window functions (taken from [Proakis, Manolakis, 1996]): Approximate transition width of main lobe Peak sidelobe [dB] -13 -27 -32 -43 -58
Type of window Rectangular Bartlett Hanning Hamming Blackman
4/L 8/L 8/L 8/L 12/L
Parameter in the Kaiser window allows to adjust the width of the main lobe, and thus also to adjust the compromise between overshoot reduction and increased transition bandwidth in the
117 118
resulting FIR lter. Magnitude frequency response 20 log10 |H (ej )| of the resulting linear-phase FIR lter, when different window functions are used to truncate the innite-length impulse response from (4.61) (desired frequency response Hd(ej ) in (4.60)):
0 |H(ej )| 20 40 60 80 100 0 0 |H(ej )| 20 40 60 80 100 0 Rectangular window 20 log10 |H(ej )| L=61, =0.2 c 0 20 40 60 80 100 0 0 20 log10 |H(ej )| 20 40 60 80 100 0 Hamming window L=61, =0.2 c
Frequency response of an FIR lter (requirement):
Hd(e ) =
L1 n=0
h(n)e
jn
With (4.62) we obtain for k = 0, . . . , L 1:
Hd(e
j 2 (k+) L )
L1 n=0
h(n)e
j 2 (k+)n/L
(4.63)
20 log
10
Multiplication of (4.63) with ej 2k/L, = 0, . . . , L 1 and summation over k = 0, . . . , L 1 yields to

0.2 0.4 0.6 / 0.8 1 Kaiser window, =4 L=61, c=0.2
0.2
0.4 0.6 /
0.8
L1 k=0
Blackman window L=61, c=0.2
j 2k/L
Hd(e
j 2 (k+) L )
20 log
10
L1 L1 k=0 n=0
h(n)e
j 2 (k+)n/L
j 2k/L
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
L1 n=0
h(n)e
j 2n/L
L1 k=0
j 2 (n)k/L
MATLAB-command for windowed linear-phase FIR design: fir1 4.4.3 Frequency sampling design Desired frequency response Hd(ej ) is specied as a set of equally spaced frequencies:
= L h() e
j 2/L
Thus, the impulse response h(n) can be obtained from the sampled desired frequency response as (n = 0, . . . , L 1):
2 k = (k + ), L
k = 0, 1, . . . , L 1,
1 . 0, 2 (4.62)
119
1 h(n) = L
L1 k=0
Hd(e
j 2 (k+)n/L j 2 (k+) L )e
(4.64)
120
Remarks:
Example:
Determine the coefcients of a type 2 linear-phase lter with length L = 32, and the desired sampled frequency response
For = 0 (4.64) is identical to the IDFT fast evaluation with IFFT In general Hd(ej L (k+)) has to be specied in amplitude and phase for every k. Exception: Type 1/3 linear-phase lters, where Hd() can be real-valued with an additional shift of the obtained impulse response (see below).
If h(n) is a real-valued sequence, the frequency response and thus also its sampled version have the Hermitian symmetry
2
1 1 2 (k+) j 2 (k+) j L 2 L Hd (e L )=e T1 0
k = 0, 1, . . . , 5, k = 6, k = 7, 8, . . . , 15.
The parameter T1 is responsible for the transition band behavior and is obtained via numerical optimization in order to reduce the magnitude of the sidelobes. The corresponding values are tabulated in the literature ([Rabiner, Gold, McGonegal, 1970], [Proakis, Manolakis, 1996]).
Hd(e
j 2 (k+) L )
= Hd (e
j 2 (Lk) L ).
Linear-phase FIR lters:
The number of frequency specications can be reduced, and (4.62) becomes 1 L odd, k = 0, 1, . . . , L 2 2 L k = (k + ), L even, k = 0, 1, . . . , 2 1 L 0, 1 .
2
For L = 32 we obtain T1 = 0.3789795 for = 0, and T1 = 0.3570496 for = 1 /2 .
Coefcients of the impulse response h(n):
(4.65)
Symmetry in h(n) can be additionally exploited such that (4.64) only has to be evaluated for n = 0, . . . , L 2 1 for L 1 even, and n = 0, . . . , L for L odd, resp. 2 Linear-phase property may be included by specifying real2 valued frequency response samples Hd(ej L (k+)) Application of (4.64) leads to a zero-phase response which has 1 to be shifted to the right by L 2 samples.
121 122
Magnitude frequency responses (20 log10 |H (ej )|):
Linear-phase lters revisited As we have shown in section 4.4.1, every linear-phase lter has a frequency response of the form
L = 32, = 0:
L = 32, = 0.5:
H ( e ) = ( j ) A( ) e
j (L1)/2
m {0, 1}, (4.66)
where A( ) is a real-valued positive or negative function (amplitude frequency response) (cmp. (4.54), (4.55), (4.56), (4.57)). It can be shown that for all types of linear-phase symmetry A( ) can always be written as a weighted sum of cosines. For example for type 1 linear-phase lters we have
MATLAB-command for the frequency sampling design of linear-phase FIR lters: fir2 4.4.4 Optimum equiripple design (Chebyshev approximation)
A( ) =
with a0 = h
(L1)/2 n=0
an cos(n )
(4.67)
Windowing design techniques (section 4.4.2) try to reduce the difference between the desired and the actual frequency response (error function) by choosing suitable windows How far can the maximum error be reduced? Theory of Chebyshev approximation answers this question and provides us with algorithms to nd the coefcients of linear-phase FIR lters, where the maximum frequency response error is minimized Chebyshev approximation: Approximation that minimizes the maximum errors over a set of frequencies Resulting lters exhibit an equiripple behavior in their frequency responses equiripple lters
L1 L1 , an = 2 h n + . (4.68) 2 2
Remaining linear-phase lters ( 2f ):
(from [Parks, Burrus: Digital Filter Design, 1987])
123
124
Problem denition Acceptable frequency response for the resulting FIR lter:
(Multiple bands are possible as well.)
Linear phase, transition bandwidth between pass- and stopband, passband deviation 1 from unity, stopband deviation 2 from zero.
then a necessary and sufcient condition that A( ) be the unique, best weighted Chebyshev approximation to a given continuous function D ( ) on F is that the weighted error function E ( ) = W ( ) (D ( ) A( )) exhibit at least r + 1 extremal frequencies in F . These frequencies are points for which 1 < < r < r+1,
E (m) = E (m+1),
and
m = 1, 2, . . . , r,
We will restrict ourselves in the following to lowpass type 1 linearphase lters. Approximation problem: Given
|E (i)| = max E ( ),
F
i = 1, . . . , r + 1.
a compact subset F of [0, ] in the frequency domain (consisting of pass- and stopband in the lowpass lter case), a desired real-valued frequency response D ( ), dened on F, a positive weight function W ( ), dened on F , and the form of A( ), here (type-1 linear-phase) r 1 A( ) = n=0 an cos(n ).
Goal: Minimization of the error
Consequences from the alternation theorem: Best Chebyshev approximation must have an equiripple error function E ( ), and is unique. Example: Amplitude frequency response of an optimum type 1 linear-phase lter with L = 13 r = 7
L = 13
A( ) =
(L1)/2 n=0
an cos(n )
Emax = max W ( ) |D ( ) A( )|
F
(4.69)
p s
over an by the choice of A( ). Alternation theorem (without proof): If A( ) is a linear combination of r cosine functions,
A( ) =
r 1 n=0
an cos(n ),
If the r+1 extremal frequencies were known, we could use the frequency-sampling design from above to specify the desired
125 126
values 1 1 at the extremal frequencies in the passband, and 2 in the stopband, respectively. How to nd the set of extremal frequencies? Remez exchange algorithm (Parks, McClellan, 1972)
It can be shown that the error function E ( ) = D ( )

r 1 n=0
an cos(n )
(4.70)
2. Interpolate to nd the frequency response on the entire grid of frequencies. 3. Search over the entire grid of frequencies for a larger error magnitude than k obtained in step 1. 4. Stop, if no larger magnitude error has been found. Otherwise, take the r + 1 frequencies, where the error attains its maximum magnitude as a new trial set of extremal frequencies and go to step 1. Flowchart ( f ):
can always be forced to take on some values for any given set of r +1 frequency points i, i = 1, . . . , r + 1. Simplication: Restriction to W ( ) = 1, leading to 1 = 2 = . This can be written as a set of linear equations according to
D (i) =
r 1 n=0
an cos(ni)+(1) ,
i = 1, . . . , r +1,
(4.71) from which we obtain a unique solution for the coefcients an, n = 0, . . . , r 1 and the error magnitude . In the Remez exchange algorithm {F } is usually chosen as an equally spaced grid of frequency points with the number of frequency points being approximately 10 L. The algorithm is initialized with a trial set of arbitrarily chosen frequencies T = {1, 2, . . . , r+1}. The Remez algorithm now consists of the following basic computations: 1. Solve the linear equations in (4.71), yielding an error magnitude k in the k-th iteration.
127
Example:
Choose the two coefcients d0 and d1 such that they minimize the Chebyshev error 2 max |x (d0 + d1 x1 )| (approximation of a parabola by a straight line) three extremal points
128
x[0,1]
resulting linear equations to be solved:
xi = d0 + d1 xi + (1) ,
achieves its maximum magnitude: T1 = [0.0, 0.625, 1.0] Linear equations to solve:
i = 0, 1, 2
(4.72)
3. Next trial set: T2 = [0.0, 0.5, 1.0] Linear equations to solve:
1 = 0.1171875
1 1 1
0 .0 0.625 1 .0
0.0 1 d0 1 d1 = 0.390625 0 1.0 1
1 = 0.125 = maximum error T2 is the extremal point set
1 1 1
0 .0 0 .5 1 .0
0.0 1 d0 1 d1 = 0.25 1.0 0 1
After the extremal points i are found, the coefcients an from Step 1 of the above algorithm are used to obtain the lter coefcients with (4.68). MATLAB-command for optimum equiripple design: remez Estimation of the lter length
1. Arbitrarily chosen trial set: T0 = [0.25, 0.5, 1.0] Matrix version of the linear equations in (4.72): 0.0625 1 0.25 1 d0 1 0 .5 1 d1 = 0.25 1.0 0 1 1 .0 1 2. Next trial set chosen as those three points, where the error
Given the stop- / passband ripple 1, 2, and the transition bandwidth = s p the necessary lter order N can be estimated as (Kaiser, 1974)
N =
10 log10(1 2) 13 . 2.324
(4.73)
0 = 0.0625
MATLAB-command for estimating the lter order: remezord
E (x) = x (d0 + d1 x)
129 130
Design example Design a linear-phase lowpass lter with the specications
the lter-length by one, L = 28:

Impulse response
0.6 20 log10 |H(ej )| [dB] 0 0.4 h(n) 20 40 60 80
Magnitude frequency response
1 = 0.01,
2 = 0.001,
p = 0.4,
s = 0.6.
Inserting these values into (4.73) leads to N 25.34 and rounding up (to be on the save side) yields a lter length L = N +1 = 27.
Impulse response
0.6 0 20 log10 |H(ej )| [dB] 0.4 h(n) 20 40 60 80
weighting 2/1 = 10 in the stopband.
0.2
0.2 0
10
15 n
20
25
100 0
0.2
0.4 0.6 /
0.8
Passband detail
1.01 1.005 1
0.2
0.2 0
10
15
20
25
100 0
0.2
0.4 0.6 /
0.8
|H(ej )|
0.995 0.99 0.985 0 0.2 / 0.4
Passband detail
1.01 |H(e )| 1.005 1
4.5 Design of IIR lters
0.995 0.99 0.985 0 0.2 / 0.4
In the following only design algorithms are discussed which convert an analog into a digital lter, however, there are also numerous algorithms for directly designing an IIR lter in the z-domain (frequency sampling method, least-squares design). Why starting point analog lter? Analog lter design is a well developed eld (lots of existing design catalogs). Problem can be dened in the z-domain, transformed into the s-domain and solved there, and nally transformed back into the z-domain.
132
In the passband the specications are not satised Increasing

131
Analog lter: Transfer function

M
Frequency response (ideal sampling assumed, compare (2.4)):
N ( s) = Ha(s) = D ( s)
n s
n=0 N n=0
H (j ) =
(4.74)
1 T
n=
Ha
j j
2n T
(4.76)
n sn
with the lter coefcients n, n, and the lter order N . Furthermore,

st
T should be selected sufciently small to avoid aliasing. Method is not suitable to design highpass lters due to the large amount of possible aliasing.
Suppose that the poles of the analog lter are distinct. Then the partial-fraction expansion of Ha(s) writes
N
Ha(s) =
h(t) e
dt
(4.75)
Ha(s) =
i=1
Ai , s si
(4.77)
(Laplace transform).
Note that linear-phase designs are not possible for causal and stable IIR lters, since the condition H (z ) = z
N
the Ai are the coefcients of the partial-fraction expansion, and the si denote the poles of the analog lter. Inverse Laplace transform of (4.77) yields
N
H (z
ha(t) =
i=1
Ai e
si t
t 0.
has to be satised (compare (4.53)) mirror-image pole outside the unit-circle for every pole inside the unit-circle unstable lter. 4.5.1 Filter design by impulse invariance Goal: Design an IIR lter with an impulse response h(n) being the sampled version of the impulse response ha(t) of the analog lter: h(n) = ha(nT ), n = 0, 1, 2, . . . , where T is the sampling interval.
133
Periodical sampling:
N
h(n) = ha(nT ) =
i=1
Ai e
si nT
Transfer function H (z ) h(n):
H (z ) =
n=0
h(n)z
n=0
Ai e
i=1
si nT
134
We then have
N n=0 n N
From (4.78) we then nally have
H (z ) =
i=1
Ai
si T
=
i=1
Ai 1 esiT z 1
H (z ) =
0.5 1 e(0.1j 3)T z 1
0.5 1 e(0.1+j 3)T z 1
.
=
. 1 (2 e0.1T cos(3T )) z 1 + e0.2T z 2

Analog lter: 20 log10 |Ha (j )|
1 (e0.1T cos(3T )) z 1
Thus, given an analog lter Ha(s) with poles si, the transfer function of the corresponding digital lter using the impulse invariant transform is
N
Magnitude frequency responses: Digital lter: 20 log10 |H (ej )|
H (z ) =
i=1
Ai s e iT
z 1
(4.78)
with poles at zi = esiT , i = 1, . . . , N . Note: (4.78) holds only for distinct poles, generalization to multiple-order poles possible. Example:
Convert the analog lter with transfer function
4.5.2 Bilinear transform Algebraic transformation between the variables s and z , mapping of the entire j -axis of the s-plane to one revolution of the unit circle in the z -plane. Denition:
Ha (s) =
s + 0. 1 (s + 0.1)2 + 9
into a digital lter using the impulse invariant method. Poles of Ha (s): s0,1 = 0.1 j 3 Partial-fraction expansion:
2 s= T
1 z 1 1 + z 1
(4.79)
T denoting the sampling interval.

0.5 0.5 + s + 0. 1 j 3 s + 0. 1 + j 3
135
Ha (s) =
The transfer function of the corresponding digital lter can be obtained from the transfer function of the analog lter Ha(s)
136
according to
Nonlinear mapping between and (warping of the frequency axis) according to 2 T 1 z 1 1 + z 1 = Ha(s). = 2 tan(/2), T = 2arctan(T /2).
(4.81)
H (z ) := Ha
Properties
Solving (4.79) for z yields z= 1 + (T /2)s , 1 (T /2)s

(4.80)
and by substituting s = + j we obtain
z=
1 + T /2 + j T /2 . 1 T /2 j T /2
< 0 |z | < 1, > 0 |z | > 1 for all causal, stable continuous-time lters map into causal stable discrete-time lters By inserting s = j into (4.80), it can be seen that |z | = 1 for all values of s on the j -axis j -axis maps onto the unit circle. Relationship between and : From (4.79) we obtain with s = j and z = ej j = = 2 T 2 T 1 ej 1 + ej j sin(/2) cos(/2) , = 2j tan(/2) T
137
Remarks:
Design of a digital lter often begins with frequency specications in the digital domain, which are converted to the analog domain by (4.81). The analog lter is then designed considering these specications (i.e. using the classical approaches from the following section) and converted back into the digital domain using the bilinear transform. When using this procedure, the parameter T cancels out and can thus be set to an arbitrary value (T = 1). Example:
Design a digital single-pole lowpass lter with a 3 dB frequency (cutoff frequency) of c = 0.2 , using the bilinear transform applied to the
138
analog lter with the transfer function
Butterworth lters
c , s + c
Ha (s) =
Lowpass Butterworth lters are allpole-lters characterized by the squared magnitude frequency response
c denoting the analog cutoff frequency. c is obtained from c using (4.81) c = 2 0.65 tan(c /2) = . T T
|H ()| =
1 , 1 + (/c)2N
(4.82)
The analog lter now has the transfer function
Ha (s) =
0.65/T , s + 0.65/T
N is the order of the lter, c is the 3 dB frequency (cutoff frequency). Since H (s) H (s)|s=j = |H (j )|2 we have from (4.82) by analytic continuation into the whole s-plane H (s) H (s) = Poles of H (s) H (s): s2 1/N j (2n+1)/N = (1) =e 2 c s,n = c e
j/2
which is transformed back into the digital domain using the bilinear transform in (4.79), leading to the transfer function of our desired digital lter: 0.245 (1 + z 1 ) . H (z ) = 1 0.509 z 1 Note that the parameter T has been divided out. The frequency response is
1 . N 1 + (s2/2 c)
H (e
)=
0.245 (1 + ej ) , 1 0.509 ej
j (2n+1)/(2N )
especially we have H (ej 0 ) = 1, and |H (ej 0.2 )| = 0.707, which is the desired response.
n = 0, . . . , 2N 1 (4.83)
4.5.3 Characteristics of commonly used analog lters
Design of a digital lter can be reduced to design an appropriate analog lter and then performing the conversion from H (s) to H (z ). In the following we briey discuss the characteristics of commonly used analog (lowpass) lters.
From (4.83): The 2N poles of H (s) H (s) occur on a circle of radius c at equally spaced points in the s-plane. The N poles for n = 0, . . . , N 1 in (4.83) are located in the left half of the s-plane and belong to H (s). The N remaining poles lie in the right half of the s-plane and belong to H (s) (stability!). Furthermore, a Butterworth lter has N zeros at .
139
140
Pole locations in the s-plane:
for the design of the corresponding digital lter obtained via bilinear transform. Example:
Determine the order and the poles of a lowpass Butterworth lter that has a 3 dB bandwidth of 500 Hz and an attenuation of 40 dB at 1000 Hz.
Frequency responses ( , |H (p)|2 = 1/(1 + 2))
3 dB frequency c = 2 fc = 1000 , stopband frequency s = 2 fs = 2000 , attenuation of 40 dB 2 = 0.01.

From (4.84) we then obtain
N =
log10 (104 1) = 6.64 2 log10 2
In order to be on the safe side we choose N = 7.

Estimation of the required lter order N : At the stopband edge frequency s (4.82) can be written as
Properties of the resulting digital lter designed with butter for N = 7, fsamp = 8000 Hz, and the above parameters (continuous-time lter transformed by bilinear transform into the discrete-time domain): Magnitude frequency response
20 log10 |H(ej )| [dB]
Transition band
1 0.8 |H(ej )| 0.6 0.4 0.2 0 0
1 2 = 2 , 1 + (s/c)2N
which leads to
2 log((1/2 )
0 20 40 60 80
N =
1) . 2 log(s/c)
(4.84)
MATLAB commands: buttord for order estimation, butter

141
100 0
0.1
0.2
0.3 0.4 /
0.5
0.6
0.05
0.1 0.15 /
0.2
0.25
142
Phase response
0
Pole/zero locations
1
The Chebyshev polynomials can be obtained by the recursive equation
arg H(ej ) [rad]
0.5 5 Im{z} 0
TN +1(x) = 2 x TN (x) TN 1(x),

Examples:
1 0.5 0 Re{z} 0.5 1
N = 1, 2, . . .
10
0.5
1 15 0 0.2 0.4 0.6 / 0.8 1
Chebyshev lters Two types of Chebyshev lters:
T2(x) = cos(2 cos1(x)) = 2cos2(cos1x) 1 = 2 x2 1
T0(x) = 1, T1(x) = cos(cos1(x)) = x
Type 1 lters are all-pole lters with equiripple behavior in the passband and monotonic characteristic (similar to a Butterworth lter) in the stopband. Type 2 lters have poles and zeros (for nite s), and equiripple behavoir in the stopband, but a monotonic characteristic in the passband.
Type 1 Chebyshev lter: Squared magnitude frequency response:
T3(x) = cos(3 cos1(x)) = 4 cos3(cos1x) 3 cos(cos1(x)) = 4 x3 3 x . . .
TN (x) represents a polynom of degree N in x. Chebyshev behavior (minimizing the maximal error) in the passband (or in the stopband for type 2 lters).
The lter parameter in (4.85) is related to the passband ripple: For N odd, TN (0) = 0 |H (0)|2 = 1, 1 for N even, TN (0) = 1 |H (0)|2 = 1+ 2 At the passband edge frequency = p we have TN (1) = 1, such that
|H ()| =
1 , 2 (/ ) 1 + 2 TN p
(4.85)
1 1+ 2
= 1 1
where is a parameter related to the passband ripple, and TN (x) is the N -th order Chebyshev polynomial dened as
1 1, (4.87) (1 1)2
which establishes a relation between the passband ripple 1 and the parameter . Typical squared magnitude frequency response for a Chebyshev type 1 lter:
144
TN (x) =
cos(N cos1(x)) cosh(N cosh

1
(x)) for |x| > 1.
for |x| 1,
(4.86)
143
Estimation of the lter order: Chebyshev lter only depend on the parameters N , , 2, and the ratio s/p. Using these values, it can be shown that the required order can be estimated as
log N =
2 + 1 2
2 (1 + 2 ) 1 2
/( 2 ) .
(4.89)
log s/p +
MATLAB commands: Type 2 Chebyshev lter: Squared magnitude frequency response:
(s/p)2 1
|H ()| =
1 2 ( / )/T 2 ( /)] 1 + 2 [TN s p s N
(4.88)
Order estimation: cheb1ord for type 1, cheb2ord for type 2. Design of the corresponding digital lter, obtained from the analog version by bilinear transform: cheby1 for type 1, cheby2 for type 2.
Elliptic (Cauer) lters
contains zeros at s < and poles

Typical squared magnitude frequency response for a Chebyshev type 2 lter:
Elliptic lters have equiripple (Chebyshev) behavior in both pass- and stopband. Transfer function contains both poles and zeros, where the zeros are located on the j -axis. Squared magnitude frequency response |H ()| =
2
1 , 1 + 2 UN (/p)
(4.90)
where UN (x) denotes the Jacobian elliptic function of order N , and the parameter controls the passband ripple. Characteristic squared magnitude frequency response for a elliptic lter:
145 146
5. Multirate Digital Signal Processing

In many practical signal processing applications different sampling rates are present, corresponding to different bandwidths of the individual signals multirate systems.

Often, a signal has to be converted from one rate to another. This process is called sampling rate conversion.
Sampling rate conversion can be carried out by analog means, that is D/A conversion followed by A/D conversion using a different sampling rate D/A converter introduces signal distortion, and the A/D converter leads to quantization effects. Sampling rate conversion can also be carried out completely in the digital domain: Less signal distortions, more elegant and efcient approach.
Filter design is optimal in pass- and stopband in the equiripple sense: However, other types of lters may be preferred due to their better phase response characteristics (i.e. approximately linear-phase), for example the Butterworth lter.
Estimation of the lter order: Required order to achieve the specications with the parameters 1, 2 and p/s, (1 1 = 1/ 1 + 2, 1 2 = 1/ 1 + 2):
Topic of this chapter is multirate signal processing and sampling rate conversion in the digital domain.
5.1 Basic multirate operations 5.1.1 Sampling rate reduction Reduction of the sampling rate (downsampling) by factor M : Only every M -th value of the signal x(n) is used for further processing, i.e. y (m) = x(m M )
. x(n) . M y (m)
N =
K (p/s) K ( K (/ ) K (
1 (p/s)2)
1 (/ )2)
(4.91)
where K (x) denotes the complete elliptic integral of the rst kind (tabulated)
/2
K (x) =
0
d 1 x2 sin2
MATLAB commands: ellipord for order estimation, ellip for the design of the corresponding digital lter obtained via bilinear transform.
147
Example: Sampling rate reduction by factor 4

148
With (5.3) the z-transform X0(z ) can be obtained as

m= m
X0(z ) =
x0(m)z
1 M
M 1
x(m)(WM z )
(5.4)
k=0 m=
(from [Fliege: Multiraten-Signalverarbeitung, 1993])
By replacing Y (z M ) = X0(z ) in (5.4) we have for the ztransform of the downsampled sequence y (m)
M 1 k=0
In the z-domain we have
X0(z ) = X0(z ) =
m=
x(mM )z
mM
1 Y (z ) = M
M
X (zWM ).
(5.5)
m=
y (m)(z )
M
M m
= Y (z ) = Y (z ) y (m)
(5.1)
With z = ej and = M the corresponding frequency response can be derived from (5.5):
M 1 k=0
Frequency response after downsampling Starting point: orthogonality of the complex exponential sequence
Y (e
1 M
M 1 k=0
j 2km/M
1 for m = M, 0 otherwise.
Z Z,
1 )= M
X (e
j ( k2 )/M )
(5.6)
(5.2)
With x0(mM ) = x(mM ) it follows
x0(m) = x(m)
1 M
M 1 k=0
Downsampling by factor M leads to a periodic repetition of the spectrum X (ej ) at intervals of 2/M (related to the high sampling frequency). WM := e
j 2/M
WM
km
(5.3)
Example: Sampling rate reduction of a bandpass signal by M = 16 ( )

150
149
a)
|Q(e j)| |Q(e j)| |H(e j)| = |X(e j)| 0 = 3 m 8 8 |X(e j(+m))| = 3 m 8 8 2 2
Antialiasing ltering followed by downsampling is often called decimation. Specication for the desired magnitude frequency response of the lowpass antialiasing (or decimation) lter: |Hd(e )| =
j
b)
1 for | | c/M,
0 c)
0 for /M | | ,
(5.7)
where c < denotes the highest frequency that needs to be preserved in the decimated signal. Downsampling in the frequency domain, illustration for M = 2: (a) input and lter spectra, (b) output of the decimator, (c) no ltering, only downsampling (V X ):
= 3 m 8 8
16
2 32
0 2
Remark: Shifted versions of X (ej ) are weighted with the factor 1/M according to (5.6). Decimation and aliasing If the sampling theorem is violated in the lower clock rate, we obtain spectral overlapping between the repeated spectra Aliasing How to avoid aliasing? Bandlimitation of the input signal v (n) prior to the sampling rate reduction with an antialiasing lter h(n) (lowpass lter).
v (n) . h(n) x(n) . M y (m)
(from [Mitra, 2000])
(a) Bandpass spectrum X (e ) is obtained by ltering. (b) Shift to the baseband, followed by decimation with M = 16. (c) Magnitude frequency response |X (ej )| at the lower sampling rate.
(from [Vary, Heute, Hess: Digitale Sprachsignalverarbeitung, 1998]) j
151
152
More general approach: Sampling rate reduction with phase offset Up to now we have always used y (0) = x(0), now we introduce an additional phase offset into the decimation process. Example for = 2
Using (5.8) we have
x(m) = x(m)
1 M
M 1 k=0
WM
k(m)
(5.9)
and transforming (5.9) into the z-domain yields
X(z ) =
1 M 1 M
M 1
x(m)(WM z )
k k
WM
k=0 m= M 1 k=0
X (zWM ) WM .
(5.10)
The frequency response can be obtained from (5.10) by substituting z = ej and = M as
Y(e

1 )= M 1 )= M
M 1 k=0 M 1 k=0
X (e
j ( 2k)/M
) WM ,
k
(5.11)
Y(e
jM
X (e
j j 2k/M
) WM .
(5.12)
Note that y2 (m) in (c) is a formal description for the output signal of the downsampler with non-integer sample indices. The real output signal y2 () is obtained by assuming integer sample locations.
We can see that each repeated spectrum is weighted with a complex exponential (rotation) factor.
5.1.2 Sampling rate increase Increase of the sampling rate by factor L (upsampling): Insertion of L 1 zero samples between all samples of y (m)
Derivation of the Fourier transform of the output signal y (m): Orthogonality relation of the complex exponential sequence:
1 M
M 1 k=0
j 2k(m)/M
1 for m = M + , 0 otherwise.
Z Z,
(5.8)
153
u( n ) =
y (n/L) for n = L, 0
otherwise.
Z Z,
(5.13)
154
. y (m) . L u(n)
Notation: Since the upsampling factor is named with L in conformance with the majority of the technical literature in the following we will denote the length of an FIR lter with LF .
Example: Sampling rate increase by factor 4
Interpolation The inserted zero values are interpolated with suitable values, which corresponds to the suppression of the L 1 imaging spectra in the frequency domain by a suitable lowpass interpolation lter.
y (m) . L u(n) . g (n) v (n)
In the z-domain the input/output relation is
U (z ) = Y (z ).
Frequency response after upsampling From (5.14) we obtain with z = ej
(5.14)
g (n): Interpolation or antiimaging lowpass lter

Specications for the interpolation lter:
jL
U (e ) = Y (e
).
(5.15)
The frequency response of y (m) does not change by upsampling, however the frequency axis is scaled differently. The new sampling frequency is now (in terms of for the lower sampling rate) equal to L 2 .
155
Suppose y (m) is obtained by sampling a bandlimited continuoustime signal ya(t) at the Nyquist rate (such that the sampling theorem is just satised). The Fourier transform Y (ej ) can thus be written with (2.4) and = /T0 as
Y (e ) =
1 Ya T0 k=
j ( 2k) T0
156
where T0 denotes the sampling period. If we instead sample ya(t) at a much higher rate T = T0/L we have
V (e ) =
1 T
k=
Upsampling in the frequency domain, illustration for L = 2: (a) input spectrum, (b) output of the upsampler, (c) output after interpolation with the lter h(n):
Ya
j ( 2k) T j ( 2k) (T0/L)
(5.16)
L Ya = T0 k=
On the other hand by upsampling of y (m) with factor L we obtain the Fourier transform of the upsampled sequence u(n) analog to (5.15) as
U (e ) = Y (e
jL
).
Therefore, we can now state our specications for the lowpass interpolation lter:
If u(n) is passed through an ideal lowpass lter with cutoff frequency at /L and a gain of L, the output of the lter will be precisely v (n) = F 1{V (ej )} in (5.16).
|Gd ( e ) | =
L for | | c/L,
for
/L | | ,
(5.17)
5.1.3 Example: Decimation and interpolation Consider the following structure:

x(n) . h(n)
u(n)
where c denotes the highest frequency that needs to be preserved in the interpolated signal (related to the lower sampling frequency).
y (m)
v (n)
. f (n) x (n)
Input-output relation? Relation between Y (z ) and U (z ) (see (5.5)), where z is replaced

157 158
by z 1/M :
Example:
Y (z ) =
1 M
M 1 k=0
U (z
1/M
WM ),
M = 4, no aliasing:
F
which by using U (z ) = H (z )X (z ) leads to
1 Y (z ) = M
M 1 k=0
H (z
1/M
WM ) X (z
1/M
WM ).
(5.18)
With V (z ) = Y (z M ) it follows
1 V (z ) = M
and we nally have
M 1 k=0
H (zWM ) X (zWM ),
(5.19)
(z ) = F (z )Y (z M ) = 1 X M
M 1 k=0
F (z ) H (zWM ) X (zWM ).
with aliasing: (5.20)
(from [Mertins: Signal Analysis, 1999])
5.1.4 Polyphase decomposition
A polyphase decomposition of a sequence x(n) leads to M subsequences x(m), = 0, . . . , M 1, which contain

159 160
only every M -th value of x(n). Example for M = 2: Decomposition into an even and odd subsequence.
Important tool for the derivation of efcient multirate ltering structures later on.
Three different decomposition types:
x0(0) = x(0), x1(0) = x(1), x2(0) = x(2),
x0(1) = x(3), x1(1) = x(4), x2(1) = x(5),
... ... ...
Type-1 polyphase components: Decomposition of x(n) into x(m), = 0, 1, . . . , M 1 with x(m) = x(mM + ), n = mM + .
(5.21)
Type-2 polyphase components: X (z ) =

M 1 =0
(M 1)
X(z )
(5.23)
With x(m) X(z ) the z-transform X (z ) can be obtained as

M 1 =0
with X(z ) x(n) = x(nM + M 1 ) (5.24) Example for M = 3:
X (z ) =
X(z )
(5.22)
x0(0) = x(2), x1(0) = x(1), x2(0) = x(0),

x0(1) = x(5), x1(1) = x(4), x2(1) = x(3),

... ... ...
Example for M = 3:
x(n) 0 1 2 x0 (m) 0 1 2 x1 (m) 0 1 2 x2 (m) 0 1 2 m m m n
Type-3 polyphase components: X (z ) =

M 1 =0 M z X (z )
(5.25)
(z ) x with X (n) = x(nM )
(5.26)
161
162
5.1.5 Nyquist-Filters Nyquist- or L-band lters:
Properties:
Used as interpolator lters since they preserve the nonzero samples at the output of the upsampler also at the interpolator output. Computationally more efcient since they contain zero coefcients. Preferred in interpolator and decimator designs.
y (m) . L u(n) . g (n) v (n)
Impulse response of a zero-phase L-th band lter: g (Ln) = for n = 0, 0

otherwise. (5.28)
every L-th coefcient is zero (except for n = 0) computationally attractive

g (n)
Using (5.14) the input-output relation of the interpolator can be stated as V (z ) = G(z ) Y (z L). The lter G(z ) can be written in polyphase notation according to
It can be shown for = 1/L that for a zero-phase L-th band lter
L1 =0
G( z ) = G0 ( z ) + z
G1 ( z ) + + z
(L1)
GL1(z ),
where the G(z ), = 0, . . . , L 1 denote the type 1 polyphase components of the lter G(z ). Suppose now that the m-th polyphase component of G(z ) is a constant, i.e. Gm(z ) = . Then the interpolator output V (z ) can be expressed as
G(zWL) = L = 1.
(5.29)
The sum of all L uniformly shifted versions of G(ej ) add up to a constant.

z = ej
G(z )
2) G(zWL ) G(zWL L 1 ) G(zWL
V (z ) = z
Y (z ) +
L1 =0, =m
G ( z ) Y ( z ) .
(5.27) v (Ln + m) = y (n); the input samples appear at the output of the system without any distortion for all n. All in-between (L 1) samples are determined by interpolation.
163
Half-band lters Special case of L-band lters for L = 2:

164
Transfer function G(z ) = + z 1G1(z 2)
For = 1/2 we have from (5.29) for the zero-phase lter g ( n) G(z ) + G(z ) = 1. (5.30) If g (n) is real-valued then G(ej ) = G(ej ()) and by using (5.30) it follows G( e ) + G( e
j j j ( )
The convolution with the length LF FIR lter h(n) can be described as
x(n) =
LF 1 k=0
h(k) v (n k),
and the downsampling with y (m) = x(m M ). Combining both equations we can write the decimation operation according to
) = 1.
(5.31)
y (m) = G(e ) exhibits a symmetry with respect to the half-band frequency /2 halfband lter.
G(ej )
LF 1 k=0
h(k) v (m M k).
(5.32)
Visualization (M = 3):
h(k ) h(0) h(LF 1) k
v (k ) v (0) p
x(0) y (0) k
v (1 k ) v (0)
FIR linear-phase halfband lter: LF = 4 1, I N
Length is restricted to
v (2 k ) v (0)
x(1) is removed k
5.2 Structures for decimation and interpolation 5.2.1 FIR direct form realization for decimation
x(n) . M y (m)
v (3 k ) v (0)
x(2) is removed k
x(3) y (1) k
v (n) .
h(n)
165
166
Multiplication of h(n) with v (1 n) and v (2 n) leads to the results x(1) and x(2) which are discarded in the decimation process these computations are not necessary.
More efcient implementation (v (n) u(n), LF N ):
which is depicted in the following:

g (k )
k y (m)
m u(k ) v (0)
k
(a) Antialiasing FIR lter in rst direct form followed by downsampling. (b) Efcient structure obtained from shifting the downsampler before the multipliers:
The output sample v (0) is obtained by multiplication of g (n) with u(n), where a lot of zero multiplications are involved, which are inserted by the upsampling operation.
More efcient implementation (v (n) x(n), LF N ):
Multiplications and additions are now performed at the lower sampling rate. Additional reductions can be obtained by exploiting the symmetry of h(n) (linear-phase).
5.2.2 FIR direct form realization for interpolation
u(n) . g (n) v (n)
y (m) .
The output v (n) of the interpolation lter can be obtained as
v ( n) =
LF 1 k=0
g ( k ) u( n k ) ,
167
(a) Upsampling followed by interpolation FIR lter in second direct form (b) Efcient structure obtained from shifting the upsampler behind the multipliers:
Multiplications are now performed at the lower sampling

168
rate, however the output delay chain still runs in the higher sampling rate. Zero multiplications are avoided. Additional reductions can be obtained by exploiting the symmetry of h(n) (linear-phase). 5.3 Decimation and interpolation with polyphase lters 5.3.1 Decimation
(a): Decimator with decimation lter in polyphase representation (b): Efcient version of (a) with M times reduced complexity Remark: The structure in (b) has the same complexity as the direct form structure from Section 5.2.1, therefore no further advantage. However, the polyphase structures are important for digital lter banks which will be discussed later on. Structure (b) in time domain (v (n) u(n)):
From Section 5.1.4 we know that a sequence can be decomposed into polyphase components. Here type-1 polyphase components (5.21) are considered in the following. Type-1 polyphase decomposition of the decimation lter h(n): The z-transform H (z ) can be written according to (5.22) as H (z ) =
M 1 =0
H(z ),
(5.33)
M denoting the downsampling factor and H(z ) h(m) the z-transform of the type-1 polyphase components h(m), = 0, . . . , M 1.
Resulting decimator structure (V (z ) U (z )):

5.3.2 Interpolation Transfer function of the interpolation lter can be written analog
169 170
to (5.33) for the decimation lter as
z 1 for every sampling rate in a multirate system in the following (instead of

introducing a special z for each sampling rate as in the sections before).
L
G( z ) =
L1 =0
G ( z ) ,
L denoting the upsampling factor, and g(m) the type-1 polyphase components of g (n) with g(m) G(z ).
Resulting interpolator structure (V (z ) X (z )):

In practice often there are applications where data has to be converted between different sampling rates with a rational ratio. Noninteger (synchronous) sampling rate conversion by factor L/M : Interpolation by factor L, followed by a decimation by factor M ; decimation and interpolation lter can be combined:

Magnitude frequency responses:
(a): Interpolator with interpolation lter in polyphase representation (b): Efcient version of (a) with L times reduced complexity As in the decimator case the computational complexity of the efcient structure in (b) is the same as for the direct form interpolator structure from Section 5.2.2. 5.4 Noninteger sampling rate conversion
Notation: For simplicity a delay by one sample will be generally denoted with
171
Efcient conversion structure In the following derivation of the conversion structure we assume
172
a ratio L/M < 1. However, a ratio L/M > 1 can also be used with the dual structures. 1. Implementation of the lter G(z ) in polyphase structure, shifting of all subsamplers into the polyphase branches:
(c) Finally, if M and L are coprime, it can be shown that upand downsampler may be exchanged in their order:
.

(d) In every branch we now have a decimator (marked with the dashed box), which can again be efciently realized using the polyphase structure from Section 5.3.1. Thus, each type-1 polyphase component g(n) is itself decomposed again in M polyphase components g(n) G(z ), = 0, . . . , L 1, = 0, . . . , M 1. Resulting structure:
2. Application of the following structural simplications: (a) It is known that if L and M are coprime (that is they have no common divider except one) we can nd 0, m0 I N such that
0 L m0 M = 1 (diophantic equation)
(5.34)
delay z in one branch of the polyphase structure can be replaced with the delay z (0 Lm0 M )
.

(b) The factor z 0L can be shifted before the upsampler, and the factor z m0M behind the downsampler:
.

Delays z m0 are realized with the output delay chain.

173 174
The terms z 0 are noncausal elements: In order to obtain a causal representation, we have to insert the extra delay block z (L1)0 at the input of the whole system, which cancels out the negative delays z 0 . Polyphase lters are calculated with the lowest possible sampling rate. L/M > 1 is realizable using the dual structure (exchange: input output, downsamplers upsamplers, summation points branching points, reverse all branching directions)
Example for L = 2 and M = 3: Application: Sampling rate conversion for digital audio signals from 48 kHz to 32 kHz sampling rate
presented methods can easily be extended to band- or highpass lters. 5.5.1 Filtering with lower sampling rate If the stopband edge frequency of a lowpass lter is substantially smaller than half of the sampling frequency, it may be advisable to perform the ltering at a lower sampling rate:
.
Hd(z ) is the (input) decimation lter, the actual ltering is carried out with the core lter Hc(z ) in the M -times lower sampling rate with sampling frequency fs1 = fs/M , and after upsampling the output signal is interpolated with Hi(z ) Single-stage implementation
Stopband- and passband edge frequencies of the decimation and interpolation lters have to be adjusted to the lter specications of the core lter:

Polyphase lters are calculated with 16 kHz sampling rate compared to 96 kHz sampling rate in the original structure. Rate conversion from 32 kHz to 48 kHz: Exercise! 5.5 Efcient multirate ltering In the following we only consider lowpass ltering, however, the
175
pass
stop
stop
Stop- and passband edge frequencies fstop and fpass of the core lter Hc(z ) are identical with those for the overall lter H (z ) = Y (z )/V (z ).
176
The interpolation lter can be chosen identical to the decimation lter, since then it is guaranteed that all imaging components are in the stopband region of the interpolation lter. Transition bandwidth for H (z ) is M -times smaller than for Hc(z ) design with a fairly small number of coefcients for Hc(z ) possible (compared to a direct design of H (z )). Stopband ripple 2 for the overall lter H (z ):
2 =
Stopband edge frequency for the decimation lter then has to be chosen less or equal than (fs1 fstop).
5.5.2 Interpolating FIR (IFIR) lters Alternative to multirate lters with decimation and interpolation, also suitable for very narrowband lters. Principle:
.
2,c (1 + 1,i )(1 + 1,d ) 2,c , fstop f (fs1 fstop ), 2,c 2,d 2,i , (fs1 fstop ) < f fs (5.35)
No real multirate lter since both lters are calculated with the same (input) sampling rate. Multirate technique is applied to the coefcients of the impulse response h(n). Realization of G(z K ) in the rst direct structure:
v (n) z K z K z K z K z K z K g (0) g (1) g (2) g (3) g (LF 2) g (LF 1)
where the approximation for fstop f (fs1 fstop) holds for small decimation and interpolation lter passband ripples 1,d and 1,i.
y (n)
Passband ripple 1 for H (z ): 1 + 1 = (1 + 1,c) (1 + 1,d) (1 + 1,i), (5.36)

approximation
1 1,c + 1,d + 1,i,
(5.37)
G(z ) is a function where all z are replaced by z K , which is equivalent to inserting K 1 zeros between the coefcients of G(z ). G(ej ) G(ejK ): Frequency response G(ej ) is compressed by factor K , K 1 imaging spectra are present:
|G(ej )|
where the last approximation is valid for small passband ripples 1,c, 1,d, and 1,i.
|G(e
jK
)|
Complexity savings (#multiplications and #additions) can be obtained by roughly a factor of 100. An even higher gain can be achieved by multistage implementations.
177
178
Furthermore, the transition bandwidth and the width of the passband for G(ejK ) are K -times smaller than for the original lter G(ej ) with the same number of lter coefcients.
the interpolation lter F (z ):
p,F = p,
s,F =
2 s. K
(5.39)
The lter F (z ) removes the imaging spectra, and H (z ) = G(z K ) F (z ) only consists of the baseband part of G(ejK ).
Design: Starting point for the design: Passband and stopband edge frequencies s, p for the overall lter H (z ) search for a suitable factor K leading to a less complex interpolation lter f ( n) . Filter specications (H, F, G are all magnitude frequency responses)
Passband ripple 1 for H (z ): 1 + 1 = (1 + 1,G) (1 + 1,F ).

(5.40)
Small passband ripples 1,G for G(z ) and 1,F for F (z ), resp., lead to the simplication
1 1,G + 1,F . Stopband ripple 2 for H (z ): 2 = 2,G (1 + 1,F ) for s s,F , 2,F (1 + 1,G) for s,F < .
(5.41)
(5.42)
Kp
Ks
For small passband ripples 1,G, 1,F we have approximately
2 2,F = 2,G.
p s
(5.43)
Example:
Design a lowpass IFIR lter with the following specications:
d = 0.05,
s = 0.1,
Requirements for passband and stopband edge frequency of the prototype G(z ): p,G = K p, s,G = K s.
(5.38)
20 log10 (1 + 1 ) = 0.2 dB 1 0.023, 20 log10 (|2 |) = 40 dB

1. We select a factor K = 4: Prototype G(z ) has the parameters
p,G = 0.2,
s,G = 0.4,
1G 0.0116 20 log10 (1+ 1,G ) 0.1 dB, 20 log10 (|2,G |) = 40 dB,

180
Requirements for passband and stopband edge frequency of

179
ripple is equally distributed between G(z ) and F (z ), see (5.41). 2. We use an linear-phase FIR Chebyshev design and insert these values for G(z ) into (4.73), yielding a lter order N = 19. However, several test designs show that the above specications are only met for N = 21, leading to a lter length of LF = N +1 = 22. Impulse response
0.4 0 20 log10 |G(ej)| [dB] 0.3 0.2 g(n) 0.1 0 0.1 0.2 0 5 10 n 15 20
Impulse response
0.4 0.3 0.2 g1(n) 0.1 0 0.1 0.2 0 20 40 n 60 80 20 log10 |G(ejK)| [dB]

0
20
40
60
20
80 0
0.2
0.4 0.6 /
0.8
40
5. Final IFIR lter h(n) = g1 (n) f (n): Magnitude frequency response

0
Passband detail
60 1.02 1.01 |H(ej )| 1 20 log10 |H(ej )| [dB]
80 0
0.2
0.4 0.6 /
0.8
20
3. Specications for the interpolation lter F (z ):
40
p,F = p = 0.05,
s,F =
2 s = 0.4, K 20 log10 (|2,F |) = 40 dB.
0.99 0.98
60
20 log10 (1 + 1,F ) 0.1 dB,
80 0
0.2
0.4 0.6 /
0.8
0.02
0.04 /
0.06
resulting lter order ((4.73)) N = 12:

Impulse response
0.3 0 0.2 f(n) 20 log10 |F(ej)| [dB]
6. H (z ) in the IFIR realization has an overall length of 35 coefcients. On the other hand we obtain from (4.73) an estimated length of 65 coefcients for the direct form implementation.
5.6 Application: Oversampled A/D and D/A converter 5.6.1 Oversampled A/D converter Extension of the A/D and D/A conversion concepts from Section 2. Structure:
0.2 0.4 0.6 / 0.8 1
20
0.1
40
60
0.1 0
6 n
10
12
80 0
x(t)
R C
x(nT /2L ) = xL (n) T /2

L
Q
b bit
Q[xL (n)]
4. Upsampling of the impulse response g (n) by factor K g1 (n) G(ejK ):
= 4
h(n)
y (n)
2L
y (n)
A/D converter
181
182
Continuous-time input signal is band-limited by the analog lowpass such that the frequency u represents the maximum frequency in the interesting frequency range for the input signal:
Sampling with a sampling frequency 2 s 2 u, L {0, 1, 2, . . . }, after the analog ltering. Here s denotes the lowest possible sampling frequency in order not to violate the sampling theorem: s = 2 u = 2/T .
L
The ltering with the lowpass h(n) now reduces the quantization noise variance by factor 2L, since
2 (L,b)
1 = 2
/2L
ee (e ) d =
/2L
(L)
22b+2 . 12 2L
(5.44)
A/D converter here is idealized as concatenation of sampler and quantizer.
After A/D conversion a lowpass ltering is carried out where the lowpass has the idealized frequency response |H (e )| =
j
Reduction of the noise variance due to oversampling by factor 2L: 2 e = 10 log10(2L) = L3.01 dB. Gain = 10 log10 2 e
(L,b)
0 otherwise.
1 for | | < /2L,
The resulting bandlimited signal can then be downsampled by factor 2L. Quantization noise variance of a b-bit midtreat quantizer according to (2.27), where the range R of the quantizer is chosen as R = 2: 22b+2 2 e = . 12 2 As an alternative e can also be obtained via the power spectral density (power spectrum) ee(ej ) as
2 e
(5.45) An increase of the oversampling factor 2L by factor 2 leads to an increase in quantizer resolution by half a bit (compare (2.29)) and to an increase of the signal-to-noise ratio (SNR) in (2.29) by 3 dB! Visualization: j
ee (e )
2 e
(a)
11111111111111 00000000000000 00000000000000 11111111111111 00000000000000 11111111111111 00000000000000 11111111111111 00000000000000 11111111111111
0 ee (ej )
2 e (L)
(b)
1 = 2
2 L
11 00 00 11 00 11 00 11 00 11 00 11 00 11 00 0011 11 00 11
0
2L
ee(e ) d.
183
Gain in (5.45) can also be used for reducing the quantizer wordlength while keeping the SNR constant:
184
Reduction of the hardware complexity of the core A/D converter Extreme case: Quantizer wordlength is reduced to b = 1 bit only a simple comparator is necessary
Requirements:
2 e (L,bL ) (0,b0 )
Analysis:
[xL(n) e y (n) = [xL(n)]Q = Q (n 1)], = xL(n) e (n 1) + e (n), (z )(1 z 1). Y (z ) = XL(z ) + E = xL(n)+ e (n) (1 (n 1)) (5.47)
(5.48)
2 e
2 ! = 1 = L 2b +2 2 2 0
2bL +2
L bL = b0 , (5.46) 2
Therefore, the z-transform of the overall quantization error sequence can be expressed as
with bL denoting the quantizer wordlength for a given L leading to the same quantization noise variance as b0 for L = 0, bL b0. Example:
Given a b0 = 16 bit A/D converter, where the analog input signal is sampled at the Nyquist rate. Choose the parameter L in the oversampled A/D converter from above such that the same quantization noise variance for bL = 1 bit is achieved. From (5.46) we obtain L = 30, leading to an oversampling factor of
(z ) 1 z 1 , E (z ) = E
which leads to the quantization noise power spectrum
ee(e ) = e e (e )|1 e
2b+2
j 2
| .
j 2 With e (noise power spectrum of a b-bit e (e ) = 12 midtreat quantizer with range R = 2) we have j
2L 109 .
Improvement: Shaping of the quantization noise The quantizer in the above block is now replaced by the following structure:
ee(e ) =
6 5 |ee(e )| x 10
3
22b+2 (1 cos( )). 6

1/3 22b+2
(5.49)
xL (n)
x L (n)
[xL (n)]Q
4 3 2 1 0 0 0.2 0.4 0.6 / 0.8 1
e (n)
z 1
185
186
Quantization noise power spectrum now has highpass character noiseshaping The noise variance after lowpass ltering with h(n) in the above oversampled A/D converter structure is now given with (5.49) as 1 = 2
/2L
Improved noiseshaping by other techniques, where the quantization noise power spectrum shows even stronger highpass character (sigma-delta modulation, more selective shaping lters).
5.6.2 Oversampled D/A converter Structure:
2 e (L,b)
ee(e ) d,
/2L
x(n) b bit
2L
x L (n) b bit
h(n)
xL (n)
Q[xL (n)] b1 < b bit
h1 (n)
22b+2 12
L+1
2 L sin(2 )
(5.50)
y (t)
C
b1 bit
DAC
Reduction of the noise variance due to oversampling by factor 2L: Gain = 10 log10
L+1
2 L sin(2 )
(5.51)
1. Input signal sampled with b bits is upsampled by factor 2L and then interpolated with h(n). 2. The resulting signal xL(n) is requantized to a wordlength of b1 < b bits, leading to a worse SNR due to higher quantization noise. 3. Filtering by h1(n) removes the quantization noise in the unused spectral regions, which increases the SNR again.
For L = 4: Gain 31 dB (compared to 12 dB without noise shaping, see (5.45)). Reducing the quantizer wordlength for constant SNR:
2 e 2 e (0,b0 )
(L,bL )
1 = 1 bL = b0+ log2 2
!
L+1
2 L sin(2 )
(5.52)
4. The b1 bit DAC in combination with a simple analog lowpass converts the signal back into the analog domain. Reason for performing a requantization: Use of a cheap low resolution D/A converter is possible, often with b1 = 1 bit. In combination with a noiseshaping approach the same SNR is obtained as when a b bit converter would be used directly on the input signal (but with higher costs).
Example:
The above example is again carried out for the noiseshaping case: For
b0 = 16 bit and bL = 1 bit we obtain from (5.52) via a computer search

(x-point iteration) L 10.47 When the input signal is sampled with A/D converter would be fsover = 211 fs 90 MHz for bL = 1.
187
fs = 44.1 kHz and b0 = 16 the new sampling frequency in the oversampled
Favored converter principle in CD players ( bitstream conversion for b1 = 1 bit)

188
5.7 Digital lter banks
A digital lter bank consists of a set of lters (normally lowpass, bandpass and highpass lters) arranged in a parallel bank. The lters split the input signal x(n) into a number of subband signals yk (n), k = 0, . . . , K 1 (analysis lter bank). Subband signals are processed and nally combined in a synthesis lter bank leading to the output signal x ( n) . If the bandwidth of the subband signal is smaller than the bandwidth of the original signal, they can be downsampled before processing processing is carried out more efciently.
. x(n) h0 (n)
u0 (n)
investigated in the following simplied structure:

x(n) H0(z ) H1(z )
.
2 2
y0 (m)
2 2
F0(z ) F1(z )
x (n)
y1 (m)
Critically subsampled lter bank: The number of subband equals the subsampling factor in every subband. Typical frequency responses for the analysis lters:
M0
y0 (m0 )
M0
v0 (n)
f0 (n)
x (n)
h1 (n)
u1 (n)
M1
y1 (m1 )
subband processing
M1
v1 (n)

f1 (n)
Frequency responses of the subband signals (for ideal lters) ( ) :

b) a) X(ej)
0 0 0 "0" 2 0 3 "0" 2 4
hK 1 (n)
uK1 (n)
MK1
yK1 (mK1 )
MK1
vK1 (n)
fK 1 (n)
analysis lter bank .
synthesis lter bank
Subband processing: Quantization (wordlength reduction) coding, (adaptive) ltering equalization, . . . 5.7.1 Two-channel lter banks: Structure and analysis Basic two-channel subband coder:
. Multiplexer
0 0
"1"
"0" 2
c)
"1" 1 0 0 2
"1" 1 3 2 4
(From [Vary, Heute, Hess: Digitale Sprachsignalverarbeitung, 1998])
(a): Magnitude frequency response of the input signal, (b) magnitude frequency response of the lowpass, and (c) highpass branch after analysis ltering and downsampling
Encoder
Demultiplexer
x(n)
H0(z ) H1(z )
2 2
Decoder
2 2
F0(z ) F1(z )
x (n)
Channel
Encoder
Decoder
Only the errors in the above system related to the lter bank are
189
Highpass subband: Baseband spectra in frequency reversed order

190
Analysis How do we have to design the lters such that
Replacing z by z 2 then leads to
x (n) = x(n D ) holds?

(D denotes the delay of the overall analysis-synthesis system in samples) We have the following relations:
Y (z ) =
1 [H (z ) X (z ) + H (z ) X (z )] , 2
where (5.53), (5.54) can be obtained by replacing H with H0 and H1, resp. 2 The connection between the subband signals and the reconstructed signal is
Y0(z ) =
1 [H0(z ) X (z ) + H0(z )X (z )] 2 1 2 Y1(z ) = [H1(z ) X (z ) + H1(z )X (z )] 2

2
(5.53) (5.54)
(z ) = Y0(z 2) F0(z ) + Y1(z 2) F1(z ) , X
(5.56)
Proof: These relations can be obtained from (5.18)
and nally by combining (5.56), (5.53), and (5.54) the inputoutput relation for the two-channel lter bank writes
Y (z ) =
1 M
M 1 k=0
H (z
1/M
WM ) X (z
1/M
WM )
(z ) = 1 [H0(z ) F0(z ) + H1(z ) F1(z )] X (z )+ X 2 1 [H0(z ) F0(z ) + H1(z ) F1(z )] X (z ). 2 (5.57) (5.57) consists of two parts: + S (z ) = [H0(z ) F0(z ) + H1(z ) F1(z )] , G(z ) = [H0(z ) F0(z ) + H1(z ) F1(z )]
(5.58) (5.59)
for M = 2. With
W2 = e
we have
j 2k/2
=e
jk
for n even,
1 for n odd,
Y (z ) = =
1 2
k=0
H ((1) z
k 1 /2
) X ((1) z
k 1 /2
S (z ): Transfer function for the input signal X (z ) through the lter bank, desirable is S (z ) = 2 z
! D
1 1 /2 1 /2 1 /2 1 /2 H (+z ) X (+z ) + H (z ) X (z ) . 2 (5.55)

191
(5.60)
192
G(z ): Transfer function for the aliasing component X (z ), desirable is (no aliasing!) G( z ) = 0 .
Two cases have now to be distinguished: 1. If G(z ) = 0, but S (z ) = c z D , then the reconstructed signal x (n) is free of aliasing, however, linear distortions are present. 2. If G(z ) = 0 and S (z ) = c z D , then we have a perfect reconstruction (PR) system, except of a scaling factor c/2 and an additional delay of D samples. 5.7.2 Two-channel quadrature-mirror lter (QMF) banks (Crosier, Esteban, Galand, 1976) Quadrature-mirror lter (QMF) banks allow the cancelation of all aliasing components, but generally lead to linear distortions (i.e. phase and amplitude distortions) Starting point: Given (lowpass) prototype H0(z ), all other lters are chosen as
!
Cancelation of all aliasing components For the linear distortion transfer function S (z ) one obtains by inserting (5.62) into (5.58) S (z ) = H0 (z ) H0 (z ),
that is, the prototype H0(z ) has to be designed such that
2 2
(5.61)
S (z ) = H0 (z ) H0 (z ) 2z
(5.64)
is satised as good as possible requirement for an ideal (constant) overall frequency response for the whole analysissynthesis system
Unfortunately, (5.64) can not be satised exactly with FIR lters, but it can be approximated with an arbitrarily small error Linear distortions can be kept small Exception: (5.64) is satised exactly by using the prototype 1 1 H0(z ) = (1 + z ) 2
(Haar lter):
F0(z ) = H0(z ),
H1(z ) = H0(z ),
F1(z ) = H1(z )
(5.62)
1 1 1 2 1 2 1 (1 + 2z + z ) (1 2z + z ) = 2z 2 2 The magnitude frequency responses of highpass and lowpass lter have for real-valued lter coefcients the mirror image property (therefore the name QMF): |H1(e
j( 2 )
Thus we have from (5.59) for the aliasing transfer function
G(z ) = H0(z ) F0(z ) + H1(z ) F1(z ) = H0(z ) H0(z ) + H0(z )(H0(z )) = 0 (5.63)
193
)| = |H0(e
j( 2 + )
)|
(5.65)
194
1.2 1 0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6 / 0.8 1 |H0(ej )| |H1(ej )|
minimization approach for a given and given prototype length LF .
Catalogs with optimized lter coefcients for h0(n) may for example be found in [Akansu, Haddad: Multirate signal decomposition, 1992]. Once a good prototype H0(z ) has been found, the remaining lters can be obtained from (5.62).
Example: (LF = 20, = 0.2 )
Impulse response h0 (n)
0.6 0.5 0.4 h0(n) 0.3 0.2 0.1 0 h1(n) 5 10 n 15 20
|H(e )|

0.5
Design of QMF banks
Usually the design is carried out by minimization of an error measure ! E = Er + Es = min. (5.66) Er refers to the linear distortion error energy
Er = 2
=0
|H0(e )| + |H0(e
j ( )
)| 1
0.1 0
0.5 0
d,
(5.67)
20 |H (ej )| 20 log |H(ej )| [dB] 0 20 40 60 80 100 0 0.2
0
10 n
15
20
Frequency responses for H0 , H1

|H (ej )|
1
Frequency response of the QMF bank

0.04 0.03 20 log10|S(e j )| [dB] 0.02 0.01 0
and Es to the energy in the stopband region of the lter
Es =
=s
10
|H0(e )| d,
(5.68)
0.01 0.02 0.03
with the stopband edge s = ( + )/2. denotes the width of the transition band, which is symmetrically centered around = /2.
0.4 0.6 /
0.8
0.04 0
0.2
0.4 0.6 /
0.8
Minimization of (5.66) can be carried out via a numerical

195 196
Efcient realization using polyphase components From (5.62) we know that H1(z ) = H0(z ) holds for the analysis highpass lter. Then, the type-1 polyphase components H0(z ) and H1(z ), {0, 1}, are related according to
yields
S (z ) = H0(z ) F0(z ) + H1(z ) F1(z ) = H0(z ) F0(z ) + (1)

+1
H0(z )F0(z ) (5.71)

!
H10(z ) = H00(z ) and H11(z ) = H01(z ).
(5.69)
This leads to an efcient realization of the analysis and synthesis lter bank, where the number of multiplications and additions can be reduced by factor four:
0
Condition for a linear distortion free system: S (z ) = 2z D With the abbreviation
T (z ) := F0(z ) H0(z )
the PR condition in (5.71) becomes
(5.72)
T (z ) + (1)
Interpretation:
+1
T (z ) = 2 z
(5.73)
(From [Mertins: Signal analysis, 1999])
5.7.3 Two-channel perfect reconstruction lter banks In order to obtain a perfect reconstruction lter bank the analysis and synthesis lters are chosen as follows ( {0, 1, 2, . . . }):
[T (z ) + T (z )] refers to the z-transform of a sequence whose odd coefcients are zero. [T (z ) T (z )]: All coefcients with an even index are zero. The PR condition in (5.71) now states that the corresponding sequences with z-transforms [T (z ) + T (z )] or [T (z ) T (z )] are allowed to have only one non-zero coefcient. Hence, for t(n) T (z ) holds 1
for
F0(z ) = z
(5.70) Aliasing transfer function: Inserting (5.70) into (5.59) yields
H1(z ),
F1(z ) = z
H0(z )
G(z ) = H0(z ) F0(z ) + H1(z ) F1(z ) = H0(z ) z =0
H1(z ) + H1(z ) z
H0(z )
n = D, = 0,
(5.74)
198
No aliasing components in the reconstructed signal!
t(n) =
Transfer function for the input signal: Inserting (5.70) into (5.58)
197
0 for n = D + 2, arbitrary otherwise.
Half-band lter (Nyquist(2) lter), see Section 5.1.5.

A half-band lter has 4 1 coefcients ( I N). Example: (a): linear-phase half-band lter (b): half-band lter with lower system delay
The zeros are {3.7321, 1.0, 1.0, 0.2679}, {1.0, 1.0},

zeros of H0 (z ) zeros of F0 (z )
such that (linear-phase lter).
H0(z ) = (1 + 2z F0(z ) = (1 + 2z
Orthogonal lter banks
+ 6z
2
+ 2z
),
+z
In the above example: Still two lters F0(z ) and H0(z ) to design in order to construct the whole PR lter bank. In an orthogonal two-channel lter bank (Smith, Barnwell, 1984), (Mintzer, 1985) it sufces to design the lowpass analysis lter H0(z ) for a PR system.
For an orthogonal two-channel bank the lter H1(z ) is chosen as
Filter design via spectral factorization A given half-band lter T (z ) can be factorized according to
H1(z ) = z
(LF 1)
H0(z
),
(5.75)
T (z ) = F0(z ) H0(z ),
i.e. one looks for the zeros of T (z ), and distributes them among the polynomials F0(z ) and H0(z ). The missing lters F1(z ) and H1(z ) can then be obtained from (5.70). General approach for the PR design of two-channel banks Example: A half-band lter is given as
LF denoting the length of h0(n). In the following we will only consider even lengths LF . Then, using (5.70) with = 0, the remaining synthesis lter can be obtained as 0(z ) = (z )(LF 1) H0(z 1), F 1(z ) = H0(z ). F
Note that (z )(LF 1) = (1)(LF 1) z (LF 1) = (1) z (LF 1) since (LF 1) odd. The factor (1) in F0(z ), F1(z ) can be regarded as a common factor multiplied to the output of the synthesis bank and can thus be removed for simplicity reasons:
{t(n)} = {1, 0, 9, 16, 9, 0, 1}.

199
F0(z ) = z
(LF 1)
H0(z
), F1(z ) = H0(z ).
(5.76)
200
Removing the factor (1) does not change anything at the aliasing cancelation property: Inserting F0(z ) and F1(z ) into (5.59) still yields G(z ) = 0. In order to obtain the condition for a analysis-synthesis system free of linear distortions we now insert (5.75) and (5.76) into (5.58), leading to
It can be easily shown that the power-complementary property also holds for h1(n), f0(n), and f1(n).
Design of orthogonal lter banks 1. Search a prototype H0(z ) which satises the powercomplementary property (5.78) or (for the interesting realvalued lters) the corresponding equation (5.77) in the zdomain. With the abbreviation TZ (z ) = H0(z ) H0(z 1)
S (z ) = z
!
(LF 1) D
(H0(z ) H0(z
) + H0(z ) H0(z
)),
= 2z
Hence, the PR condition is
TZ (z ) + TZ (z ) = 2
1
(5.79)
H0(z ) H0(z
) + H0(z ) H0(z
) = 2.
is satised, where TZ (z ) denotes a zero-phase half-band lter. (5.77)

Notation: In the following zero-phase lters and amplitude responses are denoted with the subscript Z instead of 0 to avoid confusion with the lowpass analysis lter h0 (n).
With z = ej (5.77) can be written for real-valued h0(n) according to
|H0(e )| + |H0(e
j ( + )
)| = 2
(5.78)
Valid half-band lter: TZ (z ) is a valid half-band lter if it can be factorized into H0(z ) and H0(z 1).
power-complementary property for H0(ej ).

2 |H (e )|
0 j 2
|H (e
0
j( +) 2
)|
Design goal: Find a valid half-band lter and factorize it into H0(z ) and H0(z 1). 2. When a suitable lter H0(z ) has been found, the remaining lter can be obtained from H1(z ) = z F0(z ) = z F1(z ) = z
(LF 1) (LF 1) (LF 1)
1.5
H0(z H0(z H1(z
),
(5.80)
1 1
), ) = H0(z ).
0.5
0 0
0.2
0.4 0.6 /
0.8
special case of the conditions in (5.70).

201 202
How to design a valid half-band lter? Spectral factorization may be problematic: With TZ (z ) = H0(z ) H0(z 1) and z = ej :
Visualization:
) = |H0(e )| 0 for all . (5.81) Design approach due to Smith and Barnwell, 1986 Starting point is an arbitrary linear-phase half-band lter (for example designed with the Remez algorithm from Section 4.4.4): TZ (e ) = H0(e ) H0(e A( e ) = AZ ( ) e
j j (LF 1)
(From [Fliege: Multiraten-Signalverarbeitung, 1993])
If the value
AZ ( ): Real-valued amplitude frequency response of the halfband lter a(n), AZ ( ) aZ (n), LF : length of h0(n). = min AZ ( ) < 0,
[0,2 ]
Single zeros on the unit-circle are converted to double zeros factorization into two lters is now possible
(5.82)
a non-negative amplitude frequency response can be generated with AZ + ( ) = AZ ( ) + | | . (5.83) In time-domain this corresponds with
A valid zero-phase half-band lter TZ (z ) can nally be obtained by scaling of the resulting transfer function such that ! TZ (ej/2) = 1 (note that TZ (z ) + TZ (z ) = 2 has to hold), leading to the expression
TZ (z ) =
1 AZ + ( z ) . 1 + | |
(5.85)
n = 0. (5.84) (aZ (n) denotes the zero-phase impulse response with the center of symmetry located at n = 0).
for
AZ +( ) aZ +(n) =
aZ (n) aZ (n) + | |
for
n = 0,
Remark: In practice for double zeros on the unit circle the separation process is numerically very sensitive. As a solution, the parameter | | in (5.82) and (5.83) can be enlarged by a small value zeros move pairwise off the unit-circle where due to the linearphase property they are mirrored at the unit-circle
203
204
Example Parameter for the design of TZ (z ): LF = 16, s = 0.6 , = 0.2 , = 104

h0(n)

0.8 0.6 0.4 h (n) 0.2 0 0.8 0.6 0.4 0.2 0
Pole-zero pattern TZ (z )
1 0.5 Im{z} Im{z} 0
30
Pole-zero pattern H0 (z )
0.2 0.4 0.6
1 0.5 0
15
0.8 0
10
15
0.2 0.4 0.6 0.8 0 5 n 10 15
Impulse response f0 (n)

0.8 0.6 0.4 f0(n) 0.8 0.6 0.4 f (n) 0.2 0
Impulse response f1 (n)
0.5 1 1 0 Re{z} 1 2
0.5 1 1 0 Re{z} 1 2
0.2 0
0.2 0.4 0.6
Amplitude frequency response TZ ( ) Frequency responses analysis lters

2.5 2
0.8 0
1.5
10
15
0.2 0.4 0.6 0.8 0 5 n 10 15
1
TZ ( ) 1.5 1
|H0(ej )|
|H1(ej )|=|H0(ej )|
Filter in orthogonal two-channel lter banks have an even number of coefcients since TZ (z ) = H0(z ) H0(z 1) Example:
0.5
0.5 0 0
0.2
0.4 0.6 /
0.8
0 0
0.2
0.4 0.6 /
0.8
After h0(n) has been constructed, the remaining impulse responses h1(n), f0(n), f1(n) are obtained with (5.80).
205
TZ (z ) with 7 coefcients can be factorized into two lters of length 4. The next feasible length is 11 coefcients
206
(a): Fully iterated tree lter bank, (b): Octave lter bank If the two band lter bank is of the perfect reconstruction type, the generated multiband structure also exhibits the perfect reconstruction property. which leads to two lters of length 6. 5.9 K -band lter banks In the following we briey consider K -band systems with equal subband widths and equal subsampling factors in every subband.
. x(n) h0 (n)
u0 (n)
Filter in orthogonal two-channel lter banks can not be made linear-phase (except two trivial cases) (Vaidyanathan, 1985).
5.8 Tree lter banks K channel lter banks can be developed by iterating the twochannel systems from above.
F F F
y0 (m)
v0 (n)
f0 (n)
x (n)
h1 (n)
u1 (n)
y1 (m)
v1 (n)
f1 (n)
hK 1 (n)
uK1 (n)
yK1 (m)
vK1 (n)
fK 1 (n)
F F F
.
analysis lter bank
synthesis lter bank
If K = M the analysis-synthesis system is called critically subsampled, for K > M we speak of an oversampled system. The case K < M (undersampled system) is infeasible since the sampling theorem is violated in the subbands also for ideal (brickwall) lters no reasonable reconstruction is possible.
Subset of general K -band lter banks:
F F F
Modulated lter banks

All K analysis and synthesis lters are obtained from one single analysis and synthesis prototype, resp.
207
208
Advantages:
|Hn (ej )| |H0 (ej )|
K=8 |HK1 (ej )|
More efcient implementation compared to general K -band systems. Less design complexity since only the prototype lters have to be designed. (Less storage required for the lter coefcients.)
Two important types of modulated lter banks: DFT and cosinemodulated lter banks In the following only DFT lter banks are briey considered. DFT lter banks
2 K
z-transforms: Hk (z ) = P (zWK )WK 2 ,

k kD
Fk (z ) = Q(zWK )WK 2 (5.90)
nD
Analysis and synthesis lters (k = 0, 1, . . . , K 1): hk (n) = fk ( n ) = p(n)

analysis prototype
Perfect reconstruction in the critical subsampled case is only possible for lter lengths LF = K not very good frequency selectivity. Therefore, the DFT lter bank is mainly used with M < K .
Why the name DFT lter bank? Type 1 polyphase components of the analysis prototype,
j 2 k ( n D 2) K
(5.86)
modulation and phase shifting
q ( n)
synthesis prototype
j 2 k ( n D 2) K
(5.87)
modulation and phase shifting
P (z ) =
K 1 =0
P ( z ) .
(5.91)
Frequency responses: Hk (e ) =
j j
D denotes the overall analysis-synthesis system delay, for linear phase lters we have D = LF 1.
j ( 2 k ) K ) P (e frequency shift by k 2/K j 2 k D e K 2 phase factor j 2 k D K 2
Inserting (5.91) into (5.90) then yields
Hk (z ) =
(5.88)
kD WK 2
K 1 =0
P ( z ) W K .
K 1 k=0
(5.92)
Fk (e ) =
Q(e
j ( 2 k ) K )
frequency shift by k 2/K
(5.89)
1 Analogy to the IDFT: x(n) = K
X (k ) WK
kn
phase factor
The subband lter of an DFT lter bank can be calculated with the IDFT (and thus also with the fast IFFT), where the input
209 210
signals of the IDFT are obtained by ltering with the delayed polyphase components The n-th output of the IDFT has nally to be multiplied with the rotation factors WK 2 . Structure of the analysis lter bank:
6. Spectral Estimation
In this chapter we consider the problem of estimating the power spectral density (power spectrum) of a wide-sense stationary (WSS) random process. Applications for power spectrum estimation: Signal detection and tracking, frequency estimation (e.g. for sonar or radar signals), harmonic analysis and prediction, beamforming and direction nding,. . . Problems in power spectrum estimation:
nD
(dual structure for the synthesis) If K = cM , c I N, then besides the IDFT also the polyphase ltering can be calculated in the lower sampling rate.
The amount of data is generally limited, for example, a random process may be stationary only for a short time (e.g. speech signal). On the other hand, as we have seen in Section 3.1.4 for the frequency analysis of non-random signals using the DFT, the longer the input data vector, the higher the frequency resolution of the spectral analysis. Often, only one representation of a stochastic process may be available. Therefore, an ergodic process is often assumed. Additionally, the input data may be corrupted by noise.
Estimation approaches can be classied into two classes: 1. Classical or nonparametric methods: Estimation of the autocorrelation sequence, where the power spectrum is then obtained by Fourier transform of the autocorrelation sequence. 2. Nonclassical or parametric approaches, which use a model for the random process in order to estimate the power spectrum. 6.1 Periodogram-based nonparametric methods 6.1.1 The periodogram Power spectrum vv (ej ) of a WSS random process v (n) is the
211
212
Fourier transform of the autocorrelation function vv ():
With the rectangular window
vv (e ) =
vv () e
wr ( n) =
0 otherwise,
1 n = 0, . . . , N 1,
Spectral estimation is also an autocorrelation estimation problem

From Signals and Systems we know that for a stationary random process v (n) which is ergodic in the rst and second moments the autocorrelation sequence can be theoretically obtained from the time-average
we can describe the input sequence being analyzed also as
vN ( n ) = v ( n ) w r ( n ) .
(6.4)
Then, the estimated autocorrelation sequence may be written as
vv () = lim
1 v (k + ) v (k). N 2N + 1 k=N
vv () =
(6.1)
If v (n) is only measured over a nite interval of N samples, n = 0, . . . , N 1, the autocorrelation sequence is estimated as
1 vN ()vN (). N k= (6.5) Fourier transform of the rightmost equation in (6.5) nally yields the following expression for the periodogram (VN (ej ) vN (n)): vN (k+) vN (k) =
1 N
1 vv () = N
N 1 k=0
v (k + ) v (k)
(6.2)
with the values of vv () for < 0 dened via the symmetry relation vv () = vv () = 0 for vv (), and with || N .
(per)(ej ) = 1 VN (ej ) V (ej ) = 1 |VN (ej )|2. N vv N N (6.6)

MATLAB-command: periodogram
The discrete Fourier transform of vv () from (6.2) is called periodogram (Schuster, 1898) and leads to an estimate of the power spectrum according to
(per)(ejk2/M ) = (per)(k) can be The sampled version vv vv obtained from the M1-point DFT VN (k) of vN (n): (per)(k) = 1 |VN (k)|2, vv N k = 0, . . . , M 1 1.
(6.7)
(per)(ej ) = vv
N 1 =N +1
vv ()e
(6.3)
Thus, the periodogram can be calculated very efciently using the FFT.
214
213
Example: Periodogram of white noise

2 2 If v (n) is white noise with a variance of v , then vv () = v () with a j 2 constant power spectrum vv (e ) = v .
asymptotically unbiased, which means that for N the expectation value of the estimated power spectrum is equal to the true power spectrum:
N
Sample realization for N = 32

2 1 0
Autocorrelation sequence vv ()
1 0.8 0.6
lim E
(per)(ej ) vv
= vv (e ).
(6.9)
0.4 0.2 0 0.2
1 2 3 0
On the other hand for a biased estimator there would be a difference between the expectation value and the true result. Furthermore, the estimation variance should go to zero as the data length N goes to innity:
20 0 20 40
v(n)
10
15 n
20
25
30
40
j j (per) Periodogram vv (e ) (solid), power spectrum vv (e ) (dashed)

10 5 Magnitude [dB] 0 5
(per)(ej ) lim Var vv
= 0.
(6.10)
If (6.9) and (6.10) are satised we say that the periodogram (per)(ej ) is a consistent estimate of the power spectrum.
vv
Note that there are different denitions of consistency in the literature.
Bias of the periodogram First step: Calculation of the expected value of the autocorrelation vv (). From (6.2) we have
0.2 0.4 0.6 / 0.8 1
10 15 0
Denition: Bias and consistency Desirable:
1 E { vv ()} = N = 1 N
N 1 k=0 N 1 k=0
E {v (k + ) v (k)} vv () = N vv () (6.11) N N it follows
Convergence of the periodogram to the exact power spectrum in the mean-square sense lim E (per)(ej ) vv (ej ) vv
2
= 0.
(6.8)
In order to achieve this it is necessary that the periodogram is

215
By using the symmetry relation vv () = vv () (6.11) can be written as E { vv ()} = wB () vv () with the Bartlett
216
for = 0, . . . , N 1, and for E { vv ()} = 0.
(triangular) window
Spectral resolution
N || N
wB () =
for || > N.
for || N,
(6.12)
We know from the discussion in Section 3.1.4 that the convolution with the frequency response of a window may lead to
The expected value of the periodogram can now be obtained as
(per)(ej ) vv
N 1 =N +1
E { vv ()}e
spectral smoothing, spectral leaking, the loss of the ability to resolve two nearby spectral lines.
Similarly, this also holds for the convolution between the power spectrum and the Bartlett window frequency response in (6.13). Example: (a) Power spectrum of two sinusoids in white noise, (b) expected value of the periodogram
=
which nally yields
wB () vv () e
(per)(ej ) vv
1 j j vv (e ) WB (e ) 2
(6.13)
with WB (ej ) denoting the Fourier transform of the Bartlett window 2 1 sin(N /2) j . WB (e ) = N sin(/2) Periodogram is a biased estimate, since the expected value is the convolution between the true power spectrum and the Fourier transform of the Bartlett window. Since WB (ej ) converges to an impulse for N the periodogram is asymptotically unbiased:
(from [Hayes, 1996])
lim E
(per)(ej ) vv
= vv (e )
(6.14)
Width of the main lobe of WB (ej ) increases as the data record length decreases. For a given length N there is a limit on how closely two sinusoids or narrowband processes may be located before they no longer can be resolved.
218
217
One way to dene this frequency resolution limit is to set equal to the width of the main lobe of the Bartlett window at its 6 dB point: = 0.89 2 , N
(6.15)
analog to (6.4). The periodograms of these processes are
(per)(ej ) = 1 |VN (ej )|2, vv N
which is also the frequency resolution of the periodogram. Variance of the periodogram White Gaussian random processes: It can be shown that for a white Gaussian random process v (n) the variance of the periodogram is equal to the square of the power spectrum vv (ej ) (see [Hayes, 1996]):
(per)(ej ) = 1 |XN (ej )|2. xx N (6.18) If N is large compared to the length of h(n), vN (n) can be described as vN (n) h(n) xN (n), since the transition effects can be neglected. Thus, the magnitude square frequency response |VN (ej )|2 of vN (n) can be with (6.17) expressed as |VN (e )| |H (e )| |XN (e )| = vv (e ) |XN (e )| . (6.19) Inserting (6.18) into (6.19) yields (per)(ej ) vv (ej ) (per)(ej ). vv xx
Applying the variances on both sides results in
j 2 j 2 j 2 j j 2
(per)(ej ) Var vv
2 j vv (e ).
(6.16)
(per)(ej ) Var vv
2 j (per)(ej ) , vv (e ) Var xx
Non-white Gaussian random processes: For non-white Gaussian processes, which are more important in practice, we derive an approximation for the variance of the periodogram in the following. A random process v (n) with power spectrum vv (e ) may be 2 generated by ltering white noise x(n) with variance x = 1 j with a linear lter h(n) H (e ) and
j
(per)(ej )} = 1 according to (6.16), the variance and, since Var{ xx for large N can be obtained as (per)(ej ) Var vv vv (e ).
2 j
(6.20)
Periodogram is not a consistent estimator Example:
|H (e )| = vv (e ).
(6.17)
2 For a white Gaussian noise process v (n) with v = 1 and vv (ej ) = 1 it follows from (6.13) and (6.16), resp., that
The sequences vN (n) and xN (n) are now formed by windowing

219
(per) (ej ) vv
(per) (ej ) = 1 and Var vv
= 1.
220
Thus, although the periodogram for white Gaussian noise is unbiased, the variance is independent of the data record length N .
6.1.2 Bartletts method: Periodogram averaging In contrast to the periodogram, Bartletts method (1948) provides a consistent estimate of the power spectrum. The motivation for this method comes from the fact that by averaging a set of uncorrelated measurements for a random variable v one obtains a consistent estimate of the mean E {v }. Since the periodogram is asymptotically unbiased
N = 64, overlay of 30 periodoj (per) grams vv (e )

10 0 Magnitude [dB]
N = 64, approximated periodogram variance

3 2.5 Variance 2 1.5 1 0.5
10 20 30 40 50 0 0.2 0.4 0.6 / 0.8 1
0 0
0.2
N = 256, overlay of 30 periodoj (per) grams vv (e )

10 0 Magnitude [dB]
N = 256, approximated periodogram variance

3 2.5 Variance 2 1.5 1 0.5
0.4 0.6 /
0.8
lim E
(per)(ej ) vv
= vv (e ),
it obviously sufces to nd a consistent estimate of the (per)(ej )} in order to nd a consistent periodogram E { vv estimate for the true power spectrum vv (ej ).
10 20 30 40 50 0 0.2 0.4 0.6 / 0.8 1
Estimation of the power spectrum by periodogram averaging!

Let vi(n) for i = 0, . . . , K 1 denote K uncorrelated realizations of a random process v (n) for n = 0, . . . , L 1. The periodogram of each single realization is obtained from (6.6) as
0 0
0.2
0.4 0.6 /
0.8
N = 64, periodogram average

10 0 Magnitude [dB] 10 20 30 40 50 0 0.2 0.4 0.6 / 0.8 1
N = 256, periodogram average

10 0 Magnitude [dB] 10 20 30 40 50 0 0.2 0.4 0.6 / 0.8 1
(per) (ej ) = 1 vi vi L
L1 n=0
vi ( n ) e
jn
(6.21)
The average of these periodograms is
vv (ej ) = 1 K
K 1 i=0
(per) (ej ). v v
i i
(6.22)
vv (ej ) we have with (6.22) and For the expected value of

222
221
(6.13)
Properties
vv (ej )
=E =
(per) (ej ) v v
i i
From (6.23) the expected value is
1 j j vv (e ) WB (e ). 2
(6.23)
(B )(ej ) vv
1 j j vv (e ) WB (e ) 2
(6.26)
vv (ej ) is asymptotically As with the periodogram, the estimate unbiased, i.e. for L . vv (ej )} For uncorrelated data records vi(n) the variance Var{
can be obtained in the same way from (6.22) and (6.20) as
As the periodogram the Bartlett estimate is asymptotically unbiased. The spectral resolution of the Bartlett estimate can be obtained from the resolution of the periodogram in (6.15). Since we now use sequences of length L the resolution becomes
1 2 j (e ). K vv (6.24) We can observe that the variance goes to zero if K goes to innity vv (ej ) is a consistent estimate of the power spectrum if both L and K tend to innity. vv (ej ) Var = 1 (per) (ej ) Var vi vi K
In practice: Uncorrelated realizations of a random process are generally not available, instead we have only one single realization of length N . Alternative: v (n) of length N is divided into K nonoverlapping sequences of length L with N = L K , that is vi(n) = v (n + iL), n = 0, . . . , L 1, i = 0, . . . , K 1. Thus, the Bartlett estimate is
= 0.89
2 2 = 0.89 K , L N
(6.27)
which is K times larger (worse!) than that of the periodogram. Variance: Assuming that the data sequences vi(n) are approximately uncorrelated (this is generally not the case!) the variance of the Bartlett estimate is for large N
(B )(ej ) Var vv
1 2 j (e ). K vv
(6.28)
(B )(ej ) vv
1 = N
K 1 L1 i=0 n=0
v (n + iL) e
jn
(6.25)
B ) j ( vv (e ) is a consistent estimate for K, L . The Bartlett estimate allows to trade spectral resolution for a reduction in variance by adjusting the parameters L and K . Examples: 2 = 1 of The power spectrum of a white noise Gaussian process with v length N = 256 is estimated with Bartletts method.
223
224
K = 4, L = 64, overlay of 30 B ) j ( Bartlett estimates vv (e )

K = 4, L = 64, approximated variance of Bartlett estimates

0.5 0.4 Variance 0.3 0.2 0.1 0 0
2 (n) of variance = 1,
v (n) = 3 sin(n1 ) + sin(n2 ) + (n)
(6.29)
with 1 = 0.2 , 2 = 0.25 , and length N = 512 samples. The following gures show the average power spectrum estimate over 30 realizations, and demonstrate the reduced spectral resolution of the Bartlett estimate compared to the periodogram. K = 4, L = 128, Bartlett estimate K = 8, L = 64, Bartlett estimate
30 30
10 15 0
Magnitude [dB]
K = 8, L = 32, overlay of 30 B ) j ( Bartlett estimates vv (e )

K = 8, L = 32, approximated variance of Bartlett estimates

0.5 0.4 Variance 0.3 0.2 0.1 0 0
10
Magnitude [dB] 0.2 0.4 0.6 / 30 0.8 1
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
20
20
10
10 0
10 0
0.2
0.4 0.6 /
0.8
Periodogram
10 15 0
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
Magnitude [dB]
20
10
K = 4, L = 64, Bartlett estimate average

K = 8, L = 32, Bartlett estimate average

10 0
0.2
0.4 0.6 /
0.8
6.1.3 Welchs method: Averaging modied periodograms In 1967, Welch proposed two modications to Bartletts method: 1. The data segments vi(n) of length L are allowed to overlap, where D denotes the offset between successive sequences:
0.2 0.4 0.6 / 0.8 1
10 15 0
10 15 0
0.2
0.4 0.6 /
0.8
Here, the input signal consists of two sinusoids in white Gaussian noise
225
vi(n) = v (n+iD ), n = 0, . . . , L1, i = 0, . . . , K 1. (6.30)

226
The amount of overlap between vi(n) and vi+1(n) is L D samples, and if K sequences cover the entire N data points we have N = L + D (K 1). If D = L the segments do not overlap as in Bartletts method with K = N/L.
Properties
It can be shown that the expected value of Welchs estimate is 1 j j 2 vv (e ) |W (e )| , 2LU (6.34) j where W (e ) denotes the Fourier transform of the general L-point window sequence w(n). Thus, Welchs method is an asymptotically unbiased estimate of the power spectrum. The spectral resolution of Welchs estimate depends on the used window sequence and is specied as the 3 dB width 3 dB of the main lobe of the spectral window. 3 dB is specied for some commonly used windows in the following table. E (W )(ej ) vv =
Sidelobe level [dB] -13 -27 -32 -43 -58 3 dB bandwidth 3 dB
0.89 2 L 1.28 2 L 1.44 2 L 1.30 2 L 1.68 2 L
By allowing the sequences to overlap it is possible to increase the number and/or length of the sequences that are averaged. Reduction of the variance (for larger K ) can thus be traded in for a reduction in resolution (for smaller L) and vice versa.
2. The second modication is to window the data segments prior to computing the periodogram. This leads to a so called modied periodogram
(mod)(ej ) vi vi
1 = LU
L1 n=0
vi ( n ) w ( n ) e
jn
(6.31)
with a general window w(n) of length L, and U denoting a normalization factor for the power in the window function according to L1 1 2 U = |w ( n) | . (6.32) L n=0 Welchs method may explicitly be written as
K 1 L1 i=0 n=0 2
Type of window Rectangular Bartlett Hanning Hamming Blackman
(W )(ej ) vv
1 = KLU
Remark: In (6.15) we stated the frequency resolution of the peridogram
v (n + iD ) w(n)e
jn
as the 6 dB main lobe width of the Bartlett window. Since WB (ej ) = response WR (ej ) of the rectangular window.
|WR (ej )|2 this is equivalent to the 3 dB bandwidth of the frequency
(6.33) MATLAB-command: pwelch

227
The variance of Welchs estimate highly depends on the

228
amount of overlapping. For a Bartlett window and a 50% overlap the variance is approximately
Overlay plot of 30 Welch estimates

30
Welch estimate ensemble average

30
Magnitude [dB]
(W )(ej ) Var vv
(6.35)
10
Magnitude [dB] 0.2 0.4 0.6 / 0.8 1
9 j 2 vv (e ) 8K
20
20
10
( consistent estimate). A comparison with (6.28) shows that the variance for Welchs method seems to be larger than for Bartletts method. However, for xed amount of data N and a xed resolution L here twice as many sections are averaged compared to Bartletts method. With K = 2N/L (50% overlap) (6.35) becomes
10 0 10 0 0.2 0.4 0.6 / 0.8 1
Compared to the Bartlett estimate for the same example above the use of the Hamming window reduces the spectral leakage in the estimated power spectrum. Since the number of sections (7) are about the same to those in the above example for the Bartlett estimate with K = 8, L = 64 (8 sections) both variances are also approximately the same.
(W )(ej ) Var vv
9L 2 j (e ). 16N vv
(6.36)
A comparison with (6.28) and K = N/L for the Bartlett estimate we have
6.1.4 Blackman-Tukey method: Periodogram smoothing Recall that the periodogram is obtained by a Fourier transform from the estimated autocorrelation sequence. However, for any nite data record of length N the variance of vv () will be large for values of , which are close to N . For example for lag = N 1 we have from (6.2)
(W )(ej ) Var vv
9 (B )(ej ) . Var vv 16
(6.37)
Increasing the amount of overlap yields higher computational complexity and also the correlation between the subsequences vi(n) amount of overlap is typically chosen as 50% or 75%. Example:
As an input signal we again use (6.29) which contains two sinusoids in white 2 Gaussian noise (n) of variance = 1, with 1 = 0.2 , 2 = 0.25 , and a signal length of N = 512 samples. The section length is chosen as L = 128, the amount of overlapping is 50%, and for the window we use a Hamming window.
229
vv (N 1) =
1 v (N 1) v (0). N
Two approaches for reducing the variance of vv () and thus also the variance of the peridogram: 1. Averaging periodograms and modied periodograms, resp., as utilized in the methods of Bartlett and Welch. 2. Periodogram smoothing Blackman-Tukey method (1958)
230
Blackman-Tukey method: Variance of the autocorrelation function is reduced by applying a window to vv () to decrease the contribution of the undesired estimates to the periodogram. The Blackman-Tukey estimate is given as
M
(BT )(ej ) is nonnegative for all . W (ej ) 0, such that vv Note that some of the window functions we have introduced do not satisfy this condition, for example, the Hamming and Hanning windows.
Properties
(BT )(ej ) = vv
=M
vv () w() e
(6.38)
The expected value of the Blackman-Tukey estimate can be derived for M N as 1 j j vv (e ) W (e ) 2 (6.41) j where W (e ) is the Fourier transform of the lag window. The spectral resolution of the Blackman-Tukey estimate depends on the used window. It can be shown that the variance can be approximated as E (BT )(ej ) vv = (BT )(ej ) Var vv
2 j vv (e )
where w() is a lag window being applied to the autocorrelation estimate and extending from M to M for M < N 1. Estimates of vv () having the largest variance are set to zero by the lag window the power spectrum estimate will have a smaller variance. The Blackman-Tukey power spectrum estimate from (6.38) may also be written as
(BT )(ej ) = 1 (per)(ej ) W (ej ), vv 2 vv = 1 2
(6.39)
1 N
w ().
=M
(per)(eju) W (ej (u)) du vv
(6.40)
with W (ej ) denoting the Fourier transform of the lag window. Blackman-Tukey estimate smooths the periodogram by convolution with W (ej ). Choice of a suitable window:
(6.42) From (6.41) and (6.42) we again see the trade-off between bias and variance: For a small bias, M should be large in order to minimize the width of the main lobe of W (ej ), whereas M should be small to minimize the sum term in (6.42). As a general rule of thumb, M is often chosen as M = N/5. Examples:
2 = 1 of The power spectrum of a white noise Gaussian process with v length N = 256 is estimated with the Blackman-Tukey method, where a Bartlett window with M = 51 is used.
w() should be conjugate symmetric, such that W (ej ) (and also the power spectrum) is real-valued.
231
232
6.1.5 Performance comparisons

Overlay of 30 Blackman-Tukey (BT ) j vv (e ) estimates
10 5 Magnitude [dB]
Variance
Approximated variance of the Blackman-Tukey estimates

0.5 0.4 0.3 0.2 0.1 0 0
Performance of the discussed estimators is assessed in terms of two criteria: 1. Variability V of the estimate,
0 5 10 15 0
vv (ej ) Var V =
0.2 0.4 0.6 / 0.8 1
E2
vv (ej )
(6.43)
0.2
0.4 0.6 /
0.8
Blackman-Tukey estimate ensemble average

10 5 Magnitude [dB] 0 5 10 15 0
which can be regarded as a normalized variance. 2. Overall gure of merit, which is dened as the product of the variability and the spectral resolution ,
M = V .
(6.44)
0.2
0.4 0.6 /
0.8
Results for the periodogram-based spectrum estimation techniques:

Variability V Periodogram Bartlett Welch (50% overlap, Bartlett window) Blackman-Tukey (Bartlett window of length 2M , 1 M N ) 1 Resolution Figure of merit M
In a second example we use two sinusoids in white Gaussian noise ((6.29), 2 n = 1, 1 = 0.2 , 2 = 0.25 ) for N = 512 samples. The window is a Bartlett window with M = 102.
Overlay plot of 30 BlackmanTukey estimates
30
Blackman-Tukey estimate ensemble average

30
1 K 9 8K 2M 3N
2 N 2 0.89 K N 0.89 1.28 2 L 2 M
2 N 2 0.89 N 0.89 0.72 2 N 2 N
Magnitude [dB]
10
Magnitude [dB]
20
20
10
0.64
0.43
10 0 0.2 0.4 0.6 / 0.8 1
10 0
0.2
0.4 0.6 /
0.8
233
234
Each technique has a gure of merit being approximately the same, gures of merit are inversely proportional to the length N of the data sequence. Overall performance is fundamentally limited by the amount of data being available!
6.2 Parametric methods for power spectrum estimation Disadvantages methods: of the periodogram-based (nonparametric)
difference equation is
q p
v ( n) =
i=0
b i w (n i)
i=1
ai v (n i),
(6.46)
where v (n) denotes the output and w(n) the input sequence If w(n) represents a stationary random process, then v (n) is also stationary and the power spectrum can be given as (Wiener-Lee relation) j j 2 j vv (e ) = |H (e )| ww (e ). (6.47) Parametric power spectrum estimation:
white noise source, = 0 stationary random process
2 vv (ej ) = w |H (ej )|2
Long data records are required for sufcient performance, windowing of the autocorrelation sequence limits spectral resolution. Spectral leakage effects due to windowing. A-priori information about the generating process is not exploited.
Disadvantages are removed by using parametric estimation approaches, where an appropriate model for the input process is applied. Parametric methods are based on modeling the data sequence as the output of a linear system with the transfer function (IIR lter!)
q
w(n)
H (ej )
model process v (n) ! = received process
w ai , bi
In order to estimate the power spectrum vv (ej ) we assume in our model that ww (ej ) comes from a zero-mean white noise 2 2 process with variance w . By inserting ww (ej ) = w into (6.47) the power spectrum of the observed data is
bi z i
p
vv (e ) = w |H (e )| = w ,
(6.45)
H (z ) =
i=0
|B (ej )|2 . |A(ej )|2
(6.48)
1+
i=1
ai
z i
where the ai and bi are the model parameters. The corresponding

235
Goal: Make the model process v (n) as similar as possible to the unknown received process in the mean-square error sense by adjusting the parameters ai, bi, and w the power spectrum vv (ej ) can then be obtained via (6.48).
236
In the following we distinguish among three specic cases for H (z ) leading to three different models: Autoregressive (AR) process The linear lter H (z ) = 1/A(z ) is an all-pole lter, leading to b0 = 1, bi = 0 for n > 0 in the difference equation (6.46):
p
6.2.1 Relationship between the model parameters and the autocorrelation sequence In the following it is shown that the model parameters ai, bi can be obtained from the autocorrelation sequence of the observed process v (n). These values are then inserted into (6.45) yielding H (ej ), which is then inserted into (6.48) leading to the power spectrum vv (ej ) of our observed process. In a rst step the difference equation (6.46) is multiplied by v (n ) and the expected value is taken on both sides
q
v ( n) = w ( n)
Moving average (MA) process
i=1
ai v (n i).
(6.49)
Here, H (z ) = B (z ) is an all-zero (FIR!) lter, and the difference equation becomes with an = 0 for n 0
q
E {v (n) v (n )} =
which leads to
q
v ( n) =
i=0
b i w (n i).
i=0 p
bi E {w(n i) v (n )} ai E {v (n i) v (n )},
(6.50)
Autoregressive, moving average (ARMA) process In this case the lter H (z ) = B (z )/A(z ) has both nite poles and zeros in the z-plane and the corresponding difference equation is given by (6.46). Remarks:
i=1
vv () =
i=0
bi wv ( i)
The AR model is most widely used, since the AR model is suitable of modeling spectra with narrow peaks (resonances) by proper placement of the poles close to the unit circle. MA model requires more coefcients for modeling a narrow spectrum, therefore it is rarely used as a model for spectrum estimation. By combining poles and zeros the ARMA model has a more efcient spectrum representation as the MA model concerning the number of model parameters.
237
i=1
ai vv ( i). (6.51)
The crosscorrelation sequence wv () depends on the lter impulse response:
wv () = E {v (n) w(n + )}, =E

k=0
h(k) w (n k) w(n + )
= w h().
(6.52)
238
In the last step we have used our prerequisite from above that the process w(n) is assumed to be a zero-mean white random process 2 with E {w(n k)w(n + )} = ( + k)w and known 2 variance w . Thus we have from (6.52)
may also be expressed in matrix notation according to

vv (0) (1) vv . . . vv (p 1) vv (1) vv (0) . . . vv (p 2) ... ... ... ... a1 vv (1) vv (p 1) (2) vv vv (p 2) a2 = . . . . . . . . . vv (0) ap vv (p) (6.56)
wv () =
0
2 w
for
> 0,
h() for 0.
(6.53)
By combining (6.51) and (6.53) we obtain the desired relationship for the general ARMA case:
p ai vv ( i) i=1 2 w
q
which is in short-hand notation R a = r. Once the ai have been 2 obtained by solving for the coefcient vector a, the variance w can be calculated from
p 2 w
= vv (0) +
i=1
aivv (i).
(6.57)
for > q,
p
vv () =
nonlinear relationship between the parameters vv () and ai, bi

In the following we only consider the AR model case, where (6.54) simplies to
p ai vv ( i) i=1
2 w p
ai vv ( i) for 0 q h(i) bi+ i=1 i=0 vv () for < 0. (6.54)
Since the matrix R has a special structure (Toeplitz structure) there exist efcient algorithms for solving the system of linear equations in (6.55) and (6.56), respectively (Levison-Durbin algorithm (1947, 1959), Sch ur recursion (1917)). 6.2.2 Yule-Walker method In the Yule-Walker method (also called autocorrelation method) we simply estimate the autocorrelation sequence from the observed data v (n), where the autocorrelation estimate in (6.2) is used:
for > 0, (6.55)
vv () =
i=1 vv ()
ai vv ( i) for = 0
for < 0.
1 vv () = N
N 1 n=0
v (n + ) v (n),
= 0, . . . , p.
These equations are also called Yule-Walker equations and denote a system of linear equations for the parameters ai. Equation (6.55)
239
(6.58) In the matrix version of the Yule-Walker equations (6.56) we replace vv () with vv (). The resulting linear equation system is solved for the parameter vector a, which now contains the estimated AR parameters a i, i = 1, . . . , p. Finally, we
240
2 obtain w via (6.57) from the a i and the estimated autocorrelation sequence vv ().
overlay of 30 Blackman-Tukey estimates

40 30 Magnitude [dB] 20 10 0 10 0
Blackman-Tukey ensemble average

40 30 Magnitude [dB] 20 10 0 10 0
The corresponding power spectrum estimate can now be stated from (6.48) as
(AR)(ej ) = vv 1+
2 w p 2
(6.59)
a k ejk
k=1
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
Length of observed process N = 100 (Blackman-Tukey: LB = 41):

overlay of 30 Yule-Walker estimates
40 30 Magnitude [dB] 20 10 0 10 0 Magnitude [dB] 0.2 0.4 0.6 / 0.8 1
Yule-Walker ensemble average

40 30 20 10 0 10 0
MATLAB-command: paryule 6.2.3 Examples and comparisons to nonparametric methods In the following we compute power spectrum estimates obtained with the Yule-Walker method and for comparison purposes also with the Blackman-Tukey method (Bartlett window of length LB ).
The input process consists of two sinusoids in white Gaussian noise (n) of 2 variance = 1 according to (6.29) with 1 = 0.2 , 2 = 0.25 . The model order of the Yule-Walker method is chosen as p = 50.
0.2
0.4 0.6 /
0.8
overlay of 30 Blackman-Tukey estimates

40 30 Magnitude [dB] 20 10 0
Blackman-Tukey ensemble average

40 30 Magnitude [dB] 20 10 0 10 0
Length of observed process N = 512 (Blackman-Tukey: LB = 205):

overlay of 30 Yule-Walker estimates
40 30 Magnitude [dB] 20 10 0 10 0 Magnitude [dB]
Yule-Walker ensemble average

40 30 20
10 0 10 0 10 0
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
0.2
0.4 0.6 /
0.8
We can see that only for the longer data sequence with N = 512 the resolution of the estimates are comparable. Clearly, for N = 100 the estimate based on an AR-model provides much better frequency resolution for the sinusoidal components than the Blackman-Tukey method.
242
241
Remark: Use of a certain model generally requires a-priori knowledge about the process. In case of a model mismatch (e.g. MA process and AR model) using a nonparametric approach may lead to a more accurate estimate. Example:
Consider the MA process (length N = 512)
v (n) = w(n) w(n 2),

2 where w(n) is again a white-noise zero-mean process with variance w = 1. The power spectrum of v (n) is
vv (e
) = 2 2 cos(2 ).
Ensemble average over 30 power spectrum estimates for the Yule-Walker method (AR model of order p = 4) and the Blackman-Tukey method (Bartlett window, LB = 205):
10 5 Magnitude [dB] 0 5 10 15 20 0 YuleWalker (AR(4) model) BlackmanTukey exact power spectrum 0.2 0.4 0.6 / 0.8 1
Blackman-Tukey estimate, which makes no assumption about the process, yields a better estimate of the power spectrum compared to the model-based Yule-Walker approach.
243

Course Notes Advanced DSP

Cargado por

Información del documento

Descripción original:

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Course Notes Advanced DSP

Cargado por

Copyright:

Formatos disponibles

1

Advanced Digital Signal Processing

Numerical integration and differentiation Determination of mean value, correlation, p.d.f., . . .

0.5 1 0.1 0.2 0.3 0.4 t [s] 0.5 0.6 0.7

Passing the signal through a system

Basic elements of a digital signal processing system Analog signal processing:

1.2 Digital signal processors (DSPs)

Digital signal processing:

dmin amin , dmax amax

Some currently available DSP processors and their properties (2006):

2. Digital Processing of Continuous-Time Signals

Lowpass reconstruction lter

Generation of discrete-time signals from continuous-time signals

(sifting property of the impulse function)

we nally have for the Fourier transform of xs(t)

(lengths of Dirac deltas correspond to their weighting)

(lengths of Dirac deltas correspond to their weighting)

1 for |t| < 1/2

= T = 2f T (f frequency in Hz) for the discrete signal xc (nT ) = x(n)

A0(j ) = F {a0(t)} = T sinc(T /2) e

xn(t) = xc(t) an(t) Xn(j ) =

Solution: Sampling is performed by a sample-and-hold (S/H) circuit

From (2.11) we can deduce the following:

(from [Proakis, Manolakis, 1996])

Then, with the denition of the rect() function in (2.8) we have

have both to be satised. Then,

xc(t) = xr (t) = xs(t) hr (t). (2.13)

Reconstruction of a continuous-time signal using ideal interpolation:

(from [Proakis, Manolakis, 1996])

(from [Proakis, Manolakis, 1996])

Analysis: The S/H circuit has the impulse response

which leads to a frequency response

H0(j ) = T sinc(T /2) e

|XDA(j ) = |Xc(j )| |sinc(T /2)|

Example: midtreat quantizer with L = 8 levels and range R = 8

where the amplitude xk is taken from a nite alphabet I .

Mapping in (2.19) is denoted as x (n) = Q[x(n)]

signicant bit (LSB), has the value

Number representation has no inuence on the performance of the quantization process!

The step size or the resolution of the A/D converter is given as = R 2b

Commonly used bipolar codes:

(from [Proakis, Manolakis, 1996])

(from [Proakis, Manolakis, 1996])

x (n) = x(n) + e(n)

The error sequence e(n) is modeled as a stationary white noise sequence.

(2.26) Inserting (2.23) into (2.26) yields

and by using this in (2.25) we obtain SNR = 10 log10

= 6.02 b + 10.8 20 log10

(2.25) Term () in (2.28):

2.5 Analog-to-digital converter realizations Flash A/D converter

(from [Mitra, 2000], N = b: resolution in bits)

Subranging A/D converter:

(from [Mitra, 2000], N = b: resolution in bits)

Ripple A/D converter:

2.6 Digital-to-analog converter realizations Weighted-resistor D/A converter

(from [Mitra, 2000], N = b: resolution in bits)

Output Vo of the D/A converter is given by

(from [Mitra, 2000], N = b: resolution in bits)

Iterative approach: At the k-th step of the iteration the binary

(2N 1)RL N = N VR VR , since (2 1)RL 1 (2 1)RL + 1

3. DFT and FFT

DFT: v (n) VN (k) =

1 IDFT: VN (k) v (n) = N