IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO.

6, JUNE 1998

1485

Fast and Stable Least-Squares Approach for

the Design of Linear Phase FIR Filters
Masahiro Okuda, Masaaki Ikehara, and Shin-ichi Takahashi

AbstractIn this paper, we present a new numerical algorithm

for solving the normal equations associated with the least-squares
design of linear phase FIR filters. The usual solution methods
have a computational complexity of ( 3 ). Moreover, solving
the normal equations with Gaussian elimination commonly yields
numerical errors, especially when the filter is long. Here, we
convert a least-squares method into the problem of constructing
a system of orthonormal functions. The proposed design algorithm needs only ( 2) computations, and numerical errors
can be reduced. Some examples are given to show the excellent
performance of the algorithm.

ON

ON

Index TermsComputational complexity, least-squares design,

linear-phase FIR filters, orthogonal functions.

I. INTRODUCTION

HE DESIGN of linear-phase FIR filters is an important

issue in digital signal processing. Many design methods
have been proposed by a number of authors [1][11].
The least-squares (LS) method, which minimizes the mean
squared error, is attractive to many researchers [6][8]. The
most general LS technique is one based on solving the normal
equations by Gaussian elimination. However, when a set
of linear equations is solved in practice, numerical errors
commonly arise, and it is difficult to obtain an exact LS
solution, especially when the filter is long. Moreover, solving a
(where is the number
set of linear equations requires
of parameters) computations, and thus, we need a long design
time as the order of the filter increases.
The LS approximation of linear-phase FIR filters gives rise
to a set of linear equations with the sum of a Toeplitz and
a Hankel coefficient matrix. Merchant and Parks [12] have
improved the Levinson recursion for the matrices that are
the sum of Toeplitz and Hankel coefficient matrices. This
.
algorithm has a complexity of
Vaidyanathan et al. [7] have proposed a new technique for
solving the LS problem, which is based on an eigenvalue
problem. In this algorithm, an eigenvector of an appropriate
matrix is computed to obtain the optimal filter coefficients
in the LS sense. However, this method also requires a large
amount of computation for solving the eigenvalue problem.
As is well known, the LS solution can be obtained by
means of Fourier series expansion of a desired response. In
Manuscript received June 12, 1996; revised December 10, 1997. This work
was supported by Japan Society for the Promotion of Science. The associate
editor coordinating the review of this paper and approving it for publication
was Dr. Sawasd Tantaratana.
The authors are with the Department of Electrical Engineering, Keio
University, Yokohama, Japan (e-mail: okuda@tkhm.elec.keio.ac.jp).
Publisher Item Identifier S 1053-587X(98)03911-7.

this technique, the solution can be computed easily, and the

design time is very short by virtue of the fast Fourier transform.
However, it has some drawbacks: 1) It is not possible to weight
error functions, and 2) the well-known Gibbs phenomenon
appears near cut-off frequencies.
Other closed-form expressions have been introduced by
Kaiser et al. [1], [2], which are the so-called window functions. These methods are also based on Fourier series expansion and employ a short time design. However, the obtained
filters are not optimal in any sense, and it is impossible to
specify the passband and stopband edges simultaneously.
In this paper, we propose a new approach for solving
the normal equations that arise in the design of linear-phase
FIR filters. The LS methods are converted into the problem
of constructing the orthonormal functions, and a fast and
numerically stable algorithm that produces the orthonormal
functions by means of a recurrence relation is described.
computations.
The proposed algorithm requires only
Moreover, this method improves the numerical conditioning.
In Section III, the basic theory of a linear-phase FIR filter is
briefly described. Two conventional LS methods are presented
in Section IV in order to clarify the motivation of our paper.
In Section V, the relation between a system of orthonormal
functions and LS approximation is described, and a new design
algorithm is proposed. In Section VI, several examples are
illustrated to verify the validity of the proposed algorithm,
and a comparison with the conventional methods are shown.
In the last section, we make comments on the advantages of
the algorithm.

II. LINEAR-PHASE FIR FILTERS

The magnitude response
of four types of linear-phase
FIR filters can be written in the form shown in Table I (where
is a filter length).
Table I shows that the expressions for
in all four
cases can be written as a linear combination of cosines. In this
paper, only Case 1 is considered, but a similar discussion can
easily be applied to the other cases.
We here repeat
for Case 1.

(1)

where

1053587X/98$10.00 1998 IEEE

1486

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 6, JUNE 1998

TABLE I
MAGNITUDE RESPONSE OF LINEAR PHASE FIR FILTERS

III. CONVENTIONAL WEIGHTED LEAST-SQUARES METHOD

on the discretized frequency points.

In this section, we briefly review the conventional weighted

LS (WLS) method.
Generally, the mean squared error of (1) over the interval
is defined as
(2)

(9)

In this case,

and

of (4) and (6) are rewritten as

is a weight function that is not identically zero

where
is the desired real frequency
and has positive values, and
response.
The optimal solution
in the above LS sense can
be uniquely determined by solving

(10)

(3)

We call this procedure CLS2, which is the most general

WLS method for linear-phase FIR filters. When
is high,
computing matrix is a time-consuming process. Moreover,
a set of linear equations must be solved. Then,
computations are required in addition to the calculation of
matrix entries, and numerical errors commonly arise. For
example, let us consider the design of a lowpass filter with
the width
(normalized by ) of a transition band. Burrus
et al. [10] pointed out that in the conventional techniques for
solving the LS problem, problems arise in the design of high
order filters with large . This is due to ill conditioning of
. Fig. 1 shows the magnitude response of the lowpass filter
with
, whose band edges are 0.2 and 0.35. This filter
was designed using firls.m of MATLAB, which is based on
CLS1.
It can be seen from the figure that the filter has an undesirable response in the transition band because the determinant
of
is about
. We have observed that CLS2
also has the same problem.
In the next section, we propose a new approach for the WLS
methods in order to overcome the above two problems.

as in (4), shown at the bottom of the page, and

(5)
(6)
(7)
(8)
We call this procedure of (3)(8) conventional least-squares
method 1(CLS1).
When
and
are simple functions, (7) and (8) can
be easily calculated in closed form. However, the integrals of
and
(7) and (8) are commonly difficult to derive if both
are arbitrary. In particular, when the weight function
is given by an error response, as in Lawsons algorithm
(discussed later), we can never calculate the integral in (7) and
(8). Therefore, in practice, it is efficient to define the following
cost function that is expressed as the finite sum of the errors

..
.

..
.

(11)

..

..

..
.

(4)

OKUDA et al.: FAST AND STABLE LEAST-SQUARES APPROACH FOR THE DESIGN OF LINEAR PHASE FIR FILTERS

1487

where
(18)

(19)
are orthonormal functions, the optimal soWhen
lution is given by
(20)
and
(21)

Fig. 1. Log-magnitude response designed by firls.m.

IV. LEAST-SQUARES APPROACH

BASED ON ORTHONORMAL FUNCTIONS
A. Orthogonal Functions and WLS
First, we describe the relationship between the orthonormal
(and orthogonal) functions and the WLS methods.
Let the set of functions be defined on the discretized
with the inner
frequency points
product

is
Thus, if the system of orthonormal functions
constructed, we can obtain an optimal solution without the
need for extra computations.
Therefore, the problem of solving linear equations (3) is
converted into the problem of constructing the system of orthonormal functions
. In order to solve this problem, for example, we can in principle apply GramSchmidts
orthogonalization; however, it requires the iterative computation of integrals, and it is difficult to reduce the computation
time compared with solving a normal equation with Gaussian
elimination. In the next section, we propose a fast and stable
algorithm for constructing orthonormal functions, which is
based on the Chebyshev polynomial and a recurrence relation.
B. Design Algorithm
The th order Chebyshev polynomial

(12)
The cost function in (9) is rewritten as

is defined as

Using the Chebyshev polynomials, the frequency response

can be written as

(13)
where

(22)
where

We define the following function.

..
.

(14)
Using

, the frequency response of (1) is rewritten as

Since
(15)

is orthogoAssume that the system of functions

nal for a weight function
, that is, it satisfies

basis function

, (22) is rewritten as
(23)

Then, we can define the function

(16)
Then, the optimal closed-form solution
given by

can be expressed as a polynomial of the

(24)

of (3) is
which satisfies
(17)

(25)

1488

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 6, JUNE 1998

Then

1) For
: The second and third terms
in (32) are identically zero due to the supposition of the
in the first term
mathematical induction. Since
is a polynomial whose order is smaller than , this term
,
can be expressed by the linear combination of
. Therefore, the first term is zero under the
supposition of the mathematical induction. In this case, (32)
and
.
is necessarily satisfied independent of
: The second term in (32) is zero; there2) For
fore

(26)
composes a system of orthogonal functions
If
in , we can design FIR digital filters analytically. This
function satisfies the following important theorem [13].
Theorem 1Recurrence Relation: If
is an orthogonal function that satisfies (27) and
is an orthonormal function of
, then a
thon , which satisfies
order orthonormal function
(27) with the appropriate constant values
and
, exists.
(27)
(28)
Here

(33)
must hold to satisfy (32). Then, we express
as

and

(34)
and

.
Proof: In the case of

(35)
(29)

holds according to (27). From (16),

of (33) is

Since the order of the term

can be expressed as

satisfies

(36)

(30)
Since

is normalized,

Comparing the coefficients of

the right term of (36), we obtain

is determined as

between the left and

(31)
Next, we assume that
orthonormal condition

satisfies the

, it

(37)
Equation (33) is rewritten as

(16)
where
denotes Kroneckers delta and is expressed by the
recurrence relation in (27).
expressed by (27) into (16)
Substituting

(38)
According to (27)
(39)
(32)
must be satisfied. We now prove the existence of
and satisfy (32).

and

clearly holds. Using (28) and (39), we obtain

(40)

OKUDA et al.: FAST AND STABLE LEAST-SQUARES APPROACH FOR THE DESIGN OF LINEAR PHASE FIR FILTERS

From (37), (38), and (40),

1489

satisfies

(41)
3) For

: The third term in (32) is zero, and thus

(42)
is obtained.
Therefore, with
of (41) and
of (42), the orthogonal
defined by (27) can be obtained.
function
By using the above recurrence relation, we can construct
a system of orthonormal functions very easily. Then, the frequency response is given by (20) and (21). It should be noted
that orthogonal functions can be derived using the recurrence
relation (27). However, when the orthogonal functions are used
in
instead of orthonormal functions, optimal coefficients
(17) increase exponentially, and numerical errors sometimes
arise, especially for
. In practice, this algorithm using
the orthonormal functions is more numerically stable, and an
optimal solution is obtained with little numerical error.
The flowgraph of this algorithm is shown in Fig. 2. In this
algorithm, each iteration includes six steps, and each step
computations. Since is generally selected in
requires
proportion to , this algorithm requires
computations.
A MATLAB program for designing a linear-phase FIR filter
is shown in Table II.

Fig. 2. Flowgraph of main algorithm.

we can compute the frequency response using

(44)

C. How to Obtain Filter Coefficients

Once a frequency response is obtained using the above
algorithm, filter coefficients can be calculated by the inverse
is a power of
FFT (IFFT) of the frequency response if
two, and frequency points are equally spaced over
.
is a power of two and frequency points
Similarly, when
are equally spaced over
, that is,
, the IFFT can be employed.
Even if frequency points are not equally spaced, the frequency response can be readily interpolated on equally spaced
frequency points, which are sampled by a power of two by
means of the recurrence relation. The frequency response of
orthogonal functions can be reconstructed iteratively using the
recurrence relation
(43)
which is easily derived from (27). Here,
,
, and
.
is a power
of two and a minimum number that is larger than . Then,

,
, and
already have
Note that
been derived in the main algorithm.
The computational complexity of the above steps is
. Since
is much smaller than , the computation
time required for these steps is very short compared with the
main algorithm. If is small, it is also possible to obtain the
filter coefficients easily by the IDFT.
Table III and Fig. 3 show a MATLAB program and a
flowgraph for obtaining filter coefficients (Algorithm 2), respectively.
V. EXAMPLES

AND

COMPARISON

In this section, several numerical examples are shown to

verify the advantage of the proposed algorithm. All examples
were designed on Sun/Sparc Station 20, and computer codes
were written in MATLAB. All frequencies are normalized by
, and frequency points are equally spaced. Note that all the
frequency responses and the mean squared errors were directly
calculated from filter coefficients.

1490

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 6, JUNE 1998

TABLE II
MATLAB PROGRAM OF MAIN ALGORITHM

Fig. 3. Flowgraph of Algorithm 2.

Example 1Lowpass Filter with a High Order: A low-pass

filter with a narrow transition band was designed. The filter
is 1999 (
), and the band edges are 0.1
length
and 0.105. The weight function
is 5 in the passband, 1
in the stopband, and 0 in the transition band. The grid density
of
was used. Fig. 4 illustrates the log-magnitude
response of the obtained filter. The algorithm required a total
of 18.6 s to complete the computation. Algorithm 2 in Fig. 3
was used to compute filter coefficients, which required 2.0 s.
The obtained mean squared error was
.
Comparison of Design Time to Those of Conventional
Methods: A lowpass filter with a narrow transition band
was designed. The band edges are 0.3 and 0.31. The weight
function
is 1 in the passband and stopband and 0 in
was used.
the transition band. The grid density of
Since the time required for Algorithm 2 depends on , we
obtained filter coefficients directly by means of the IDFT

instead of Algorithm 2 in order to show that the computation

time increases almost linearly with fixed .
and the computation time
In Fig. 5, the relation between
is shown. The plots marked by o, +, *, and . were
obtained using the proposed algorithm, the MATLAB function
firls.m, Gaussian elimination of the MATLAB program, and
the block Levinson algorithm [12], respectively. Note that the
block Levinson algorithm was fully coded by MATLAB.
The computer code firls.m is based on CLS1, and Gaussian elimination and the block Levinson algorithm were used
for CLS2. As mentioned above, the proposed algorithm reor
computation and the design
quires only
time is significantly reduced, especially for higher order filters.
Although CLS2 includes the calculation of at least
cosine function, the proposed algorithm requires the
calculation of only cosine functions
Example 2: In this example, ill conditioning is examined.
is 169 (
), and the band edges of the lowpass
is 1 at
filter are 0.2 and 0.35. The weight function
the passband and stopband and 0 in the transition band. The

OKUDA et al.: FAST AND STABLE LEAST-SQUARES APPROACH FOR THE DESIGN OF LINEAR PHASE FIR FILTERS

1491

TABLE III
MATLAB PROGRAM OF ALGORITHM 2

Fig. 5. Computation time versus

Fig. 6.

Fig. 4. Example 1: Log-magnitude response.

grid density of
was used. This specification is the
same as in the case of Fig. 1 in Section IV. Fig. 6 illustrates
the results of the proposed algorithm. The magnitude response
obtained using CLS2 is almost the same as that of CLS1. As is
pointed out in Section IV, CLS1 and CLS2 have a numerical
and the width of the transition band
problem when both
are large. However, the proposed method can achieve a better
magnitude response than the conventional methods.
Fig. 7 shows the plots of approximation error versus .
This error corresponds to the log of mean squared error, that is,
. Although the errors arising in CLS1 and Gaussian

plots.

Example 2. Log-magnitude response (proposed).

elimination do not decrease monotonically for lengths longer

, that in the proposed method decreases almost
than
monotonically. It is clear that numerical errors are reduced
in the proposed algorithm. Note that the error plot of the
Levinson algorithm is almost the same as for the Gaussian
elimination. In addition, the filter coefficients were obtained
using Algorithm 2.
Example 3Chebyshev Approximation: In the next example, we apply the proposed method to the Chebyshev approximation problem and compare it with the ParksMcClellan
algorithm. Lawsons algorithm is often used for the Chebyshev
approximation problem, in which the WLS problems are
solved iteratively. However, the convergence of the algorithm
is very slow. Lim et al. [9] have proposed a modified Lawsons algorithm to improve the convergence, whose weighting
function is based on the envelope of the error. In this study,
we adopt this modified Lawsons algorithm to compare our
method to the MATLAB function remez.m, which corresponds to the ParksMcClellan algorithm.

1492

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 6, JUNE 1998

Fig. 7. Example 2. Approximation error versus

1 1 1 firls.m, - - CLS2.

N . : Proposed algorithm,

Fig. 8. Example 3. Log-magnitude response.

The specifications of the lowpass filter are that

is 199
(
) and that the band edges are 0.4 and 0.42. The
weight function
is 1 in the passband and stopband and
0 in the transition band. Figs. 8 and 9 show the results of the
. The filter
designed filter with the grid density of
coefficients were obtained using Algorithm 2. The numerical
complexity of the modified Lawsons algorithm depends on
the grid density , and the filter obtained is optimal within
the grid density. In this example, five to nine iterations and
about 4.13, 8.47, 10.7, and 15.3 s were needed to achieve
convergence in the cases of
,
,
, and
,
respectively. Using remez.m, about 11.3 s were required to
complete the computation.
VI. CONCLUSION
We present a new numerical algorithm for solving the
normal equations associated with the least-squares design
of linear-phase FIR filters. In this work, we construct a

Fig. 9. Example 2. Log-magnitude response in passband.

system of orthonormal functions using the recurrence relation

of Chebyshev polynomials so that the computation time is
reduced and the numerical errors can be eliminated, especially
for the higher order filters.
In practical filter designs, permissible errors in the passband
and stopband regions are usually given as a part of the
specification of the filters. Thus, it is desirable to design
filters that have the minimum orders required to satisfy the
specified errors. The minimum orders are determined through
trial and error. In the conventional least-squares methods, we
must redesign the filter from the beginning until the desired
specifications are satisfied. However, in our new method, if
orthogonal functions have already been obtained, a
the
filter with the order
can be designed by finding only
the
th orthonormal function. Therefore, we can easily
determine the minimum filter orders within the specific errors.
We have compared the proposed algorithm with the conventional least-squares methods and the ParksMcClellan algorithm. The advantages of our algorithm were supported by
the results of the numerical examples.
ACKNOWLEDGMENT
The authors would like to thank the reviewers for useful
suggestions and comments, which improved the paper.
REFERENCES
[1] L. R. Rabiner and B. Gold, Theory and Application of Digital Signal
Processing. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[2] A. V. Oppenheim and R. W. Schafer, Digital Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[3] T. W. Parks and J. H. McClellan, Chebyshev approximation for
nonrecursive digital filters with linear phase, IEEE Trans. Circuit
Theory, vol. CT-19, pp. 189194, 1972.
[4] J. H. McClellan, T. H. Parks, and L. R. Rabiner, A computer program
for designing optimum FIR linear phase digital filters, IEEE Trans.
Audio. Electroacoust., vol. AU-21, pp. 506526, 1973.
[5] J. H. McClellan and T. H. Parks, A unified approach to the design of
optimum FIR linear-phase digital filters, IEEE Trans. Circuit Theory,
vol. CT-20, pp. 506526, 1973.
[6] V. R. Algazi and M. Suk, On the frequency weighted least squares
design of finite duration filters, IEEE Trans. Circuits Syst., vol. CAS-34,
pp. 8095, 1987.

OKUDA et al.: FAST AND STABLE LEAST-SQUARES APPROACH FOR THE DESIGN OF LINEAR PHASE FIR FILTERS

[7] P. P. Vaidyanathan and T. Q. Nguyen, Eigenfilters: A new approach to

least squares FIR filter design and applications, IEEE Trans. Circuits
Syst., vol. CAS-22, pp. 943953, 1975.
[8] V. R. Algazi, M. Suk, and C. S. Rim, Design of almost minimax FIR
filters in one and two dimensions by WLS techniques, IEEE Trans.
Circuits Syst., vol. CAS-33, pp. 590596, 1986.
[9] Y. C. Lim, J. H. Lee, C. K. Chen, and R. H. Yang, A weighted least
squares algorithm for quasiequiripple FIR and IIR digital filter design,
IEEE Trans. Signal Processing, vol. 40, pp. 551558, 1992.
[10] C. S. Burrus, A. W. Soewito, and R. A. Gopinath, Least squares error
FIR filter design with transition bands, IEEE Trans. Signal Processing,
vol. 40, pp. 13271340, 1992.
[11] C. S. Burrus, Multiband least squares FIR filter design, IEEE Trans.
Signal Processing, vol. 43, pp. 412421, 1995.
[12] G. A. Merchant and T. W. Parks, Efficient solution of a Toeplitz-plusHankel coefficient matrix system of equations, IEEE Trans. Acoust.,
Speech, Signal Processing, vol. ASSP-30, pp. 4044, 1982.
[13] A. Ralston and P. Rabinowitz, A First Course in Numerical Analysis.
New York: McGraw-Hill, 1989.
[14] E. W. Cheney, Introduction to Approximation Theory. New York:
McGraw-Hill, 1989.
[15] C. Moler, J. Little, and S. Bangerd, Matlab Users Guide. South
Natick, MA: Math Works, 1989.

Masahiro Okuda received the B.E., the M.E., and

the Dr.Eng. degrees in electrical engineering in
1993, 1995, and 1998, respectively, from Keio University, Yokohama, Japan.
He is currently working at Keio University as a
Visiting Scholar. His research interests are in the
area of filter design problems and wavelet image
coding.

1493

Masaaki Ikehara received the B.E., the M.E., and

the Dr.Eng. degrees in electrical engineering in
1984, 1986, and 1989, respectively, from Keio University, Yokohama, Japan.
From 1989 to 1992, he was a Lecturer in the
Department of Electrical Engineering and Computer
Science, Nagasaki University, Nagasaki, Japan. In
1992, he joined the Faculty of Engineering, Keio
University. He is currently working at Boston University, Boston, MA, as a Visiting Scholar. His
research interests are in the area of multirate processing, wavelet image coding, and filter design problems.

Shin-ichi Takahashi received the B.E., M.E., and

Dr.Eng. degrees in electrical engineering in 1962,
1964, and 1967, respectively, from Keio University,
Yokohama, Japan.
He is currently a Professor of Electrical Engineering at Keio University. His interests include circuit
theory and digital signal processing.